issue_comments
10,495 rows sorted by updated_at descending
This data as json, CSV (advanced)
updated_at (date) >30 ✖
- 2021-03-22 66
- 2021-11-19 60
- 2022-11-16 58
- 2020-09-22 53
- 2020-10-15 52
- 2023-08-18 52
- 2020-10-30 49
- 2022-10-26 47
- 2022-03-21 46
- 2020-06-09 43
- 2022-10-27 43
- 2022-12-13 43
- 2022-01-09 42
- 2022-10-25 41
- 2020-10-20 40
- 2020-06-18 39
- 2020-12-18 39
- 2021-11-16 39
- 2021-12-16 39
- 2022-06-14 39
- 2020-05-27 38
- 2020-12-30 38
- 2020-10-09 37
- 2022-03-19 37
- 2022-12-15 37
- 2021-11-20 36
- 2022-01-20 36
- 2023-03-08 36
- 2023-07-22 36
- 2022-11-18 35
- …
issue >30
- Redesign default .json format 55
- Show column metadata plus links for foreign keys on arbitrary query results 51
- ?_extra= support (draft) 49
- Rethink how .ext formats (v.s. ?_format=) works before 1.0 48
- Upgrade to CodeMirror 6, add SQL autocomplete 48
- JavaScript plugin hooks mechanism similar to pluggy 47
- Updated Dockerfile with SpatiaLite version 5.0 45
- Complete refactor of TableView and table.html template 45
- Port Datasette to ASGI 42
- Authentication (and permissions) as a core concept 40
- invoke_startup() is not run in some conditions, e.g. gunicorn/uvicorn workers, breaking lots of things 36
- Deploy a live instance of demos/apache-proxy 34
- await datasette.client.get(path) mechanism for executing internal requests 33
- Maintain an in-memory SQLite table of connected databases and their tables 32
- Research: demonstrate if parallel SQL queries are worthwhile 32
- Ability to sort (and paginate) by column 31
- Default API token authentication mechanism 30
- Port as many tests as possible to async def tests against ds_client 29
- link_or_copy_directory() error - Invalid cross-device link 28
- Server hang on parallel execution of queries to named in-memory databases 28
- Add ?_extra= mechanism for requesting extra properties in JSON 27
- Export to CSV 27
- base_url configuration setting 27
- Documentation with recommendations on running Datasette in production without using Docker 27
- Optimize all those calls to index_list and foreign_key_list 27
- Support cross-database joins 26
- Ability for a canned query to write to the database 26
- table.transform() method for advanced alter table 26
- New pattern for views that return either JSON or HTML, available for plugins 26
- Proof of concept for Datasette on AWS Lambda with EFS 25
- …
created_at (date) >30 ✖
- 2021-03-22 66
- 2021-11-19 60
- 2022-11-16 59
- 2020-10-15 52
- 2020-09-22 51
- 2023-08-18 51
- 2020-10-30 49
- 2022-10-26 47
- 2022-03-21 46
- 2020-12-18 43
- 2020-06-09 42
- 2022-10-27 42
- 2022-12-13 42
- 2020-06-18 41
- 2022-10-25 41
- 2020-10-20 40
- 2022-01-09 40
- 2022-06-14 40
- 2020-05-27 39
- 2021-11-16 39
- 2021-12-16 39
- 2020-12-30 38
- 2022-12-15 37
- 2023-03-08 37
- 2020-10-09 36
- 2021-11-20 36
- 2022-01-20 36
- 2022-03-19 36
- 2023-07-22 36
- 2020-09-15 34
- …
reactions 23 ✖
- {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 10,032
- {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 209
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} 40
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 37
- {"total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 20
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0} 13
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1} 9
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 1, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 7
- {"total_count": 3, "+1": 3, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 6
- {"total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 2, "rocket": 0, "eyes": 0} 3
- {"total_count": 2, "+1": 0, "-1": 0, "laugh": 0, "hooray": 2, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 3
- {"total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 3
- {"total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0} 2
- {"total_count": 3, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 3, "rocket": 0, "eyes": 0} 2
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 1, "heart": 0, "rocket": 0, "eyes": 0} 1
- {"total_count": 15, "+1": 7, "-1": 0, "laugh": 1, "hooray": 1, "confused": 0, "heart": 5, "rocket": 1, "eyes": 0} 1
- {"total_count": 2, "+1": 0, "-1": 0, "laugh": 1, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0} 1
- {"total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1} 1
- {"total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 1, "eyes": 0} 1
- {"total_count": 3, "+1": 0, "-1": 0, "laugh": 0, "hooray": 3, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 1
- {"total_count": 4, "+1": 4, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 1
- {"total_count": 5, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 3, "eyes": 0} 1
- {"total_count": 5, "+1": 5, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 1
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1730226107 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730226107 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nISe7 | simonw 9599 | 2023-09-21T20:06:19Z | 2023-09-21T20:06:19Z | OWNER | No that's not it actually, it's something else. Got to this point:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730219703 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730219703 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIQ63 | simonw 9599 | 2023-09-21T20:01:54Z | 2023-09-21T20:01:54Z | OWNER | The problem is here:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730214654 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730214654 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIPr- | simonw 9599 | 2023-09-21T19:59:51Z | 2023-09-21T19:59:51Z | OWNER | So the problem is the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730212597 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730212597 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIPL1 | simonw 9599 | 2023-09-21T19:58:38Z | 2023-09-21T19:58:38Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | ||
1730211445 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730211445 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIO51 | simonw 9599 | 2023-09-21T19:57:44Z | 2023-09-21T19:57:44Z | OWNER | In the debugger: ```
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730210728 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730210728 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIOuo | simonw 9599 | 2023-09-21T19:57:08Z | 2023-09-21T19:57:08Z | OWNER | In my Python 3.8 environment I ran:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730208566 | https://github.com/simonw/datasette/issues/2193#issuecomment-1730208566 | https://api.github.com/repos/simonw/datasette/issues/2193 | IC_kwDOBm6k_c5nIOM2 | simonw 9599 | 2023-09-21T19:55:19Z | 2023-09-21T19:55:19Z | OWNER | Yes, the new script seems to work. On Python 3.11:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run 1907655261 | |
1730206629 | https://github.com/simonw/datasette/issues/2193#issuecomment-1730206629 | https://api.github.com/repos/simonw/datasette/issues/2193 | IC_kwDOBm6k_c5nINul | simonw 9599 | 2023-09-21T19:53:39Z | 2023-09-21T19:53:39Z | OWNER |
It suggested doing this instead: ```bash !/bin/bashPLUGINS=$(datasette plugins) if ! echo "$PLUGINS" | jq 'any(.[]; .name == "datasette-json-html")' | grep -q true; then echo "Test failed: datasette-json-html not found" exit 1 fi PLUGINS2=$(DATASETTE_LOAD_PLUGINS=datasette-init datasette plugins) if ! echo "$PLUGINS2" | jq 'any(.[]; .name == "datasette-json-html")' | grep -q false; then echo "Test failed: datasette-json-html should not have been loaded" exit 1 fi if ! echo "$PLUGINS2" | jq 'any(.[]; .name == "datasette-init")' | grep -q true; then echo "Test failed: datasette-init should have been loaded" exit 1 fi PLUGINS3=$(DATASETTE_LOAD_PLUGINS='' datasette plugins) if ! echo "$PLUGINS3" | grep -q '[]'; then echo "Test failed: datasette plugins should have returned []" exit 1 fi ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run 1907655261 | |
1730203356 | https://github.com/simonw/datasette/issues/2193#issuecomment-1730203356 | https://api.github.com/repos/simonw/datasette/issues/2193 | IC_kwDOBm6k_c5nIM7c | simonw 9599 | 2023-09-21T19:51:04Z | 2023-09-21T19:51:04Z | OWNER | The script: I'm not sure why those |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run 1907655261 | |
1730202533 | https://github.com/simonw/datasette/issues/2193#issuecomment-1730202533 | https://api.github.com/repos/simonw/datasette/issues/2193 | IC_kwDOBm6k_c5nIMul | simonw 9599 | 2023-09-21T19:50:22Z | 2023-09-21T19:50:22Z | OWNER | Here's the failure in CI, which did not cause the workflow to fail even though it should have: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run 1907655261 | |
1730201226 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730201226 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIMaK | simonw 9599 | 2023-09-21T19:49:20Z | 2023-09-21T19:49:20Z | OWNER | That passed on 3.8 but should have failed: https://github.com/simonw/datasette/actions/runs/6266341481/job/17017099801 - the "Test DATASETTE_LOAD_PLUGINS" test shows errors but did not fail the CI run. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730188367 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730188367 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIJRP | simonw 9599 | 2023-09-21T19:38:28Z | 2023-09-21T19:40:38Z | OWNER | I'll imitate
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730185322 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730185322 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIIhq | simonw 9599 | 2023-09-21T19:35:49Z | 2023-09-21T19:35:49Z | OWNER | I think I can fix this using https://importlib-resources.readthedocs.io/en/latest/using.html - maybe as a dependency only installed if the Python version is less than 3.9. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730183405 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730183405 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIIDt | simonw 9599 | 2023-09-21T19:34:09Z | 2023-09-21T19:34:09Z | OWNER | Confirmed: https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730171241 | https://github.com/simonw/datasette/issues/2057#issuecomment-1730171241 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5nIFFp | simonw 9599 | 2023-09-21T19:27:25Z | 2023-09-21T19:27:25Z | OWNER | This broke in Python 3.8:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1730162283 | https://github.com/simonw/datasette/issues/2189#issuecomment-1730162283 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5nIC5r | simonw 9599 | 2023-09-21T19:19:47Z | 2023-09-21T19:19:47Z | OWNER | I'm going to release this in |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1729961503 | https://github.com/simonw/datasette/pull/2190#issuecomment-1729961503 | https://api.github.com/repos/simonw/datasette/issues/2190 | IC_kwDOBm6k_c5nHR4f | asg017 15178711 | 2023-09-21T16:56:57Z | 2023-09-21T16:56:57Z | CONTRIBUTOR | TODO: add similar checks for permissions/allow/canned queries |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Raise an exception if a "plugins" block exists in metadata.json 1901483874 | |
1728504633 | https://github.com/simonw/datasette/pull/2191#issuecomment-1728504633 | https://api.github.com/repos/simonw/datasette/issues/2191 | IC_kwDOBm6k_c5nBuM5 | simonw 9599 | 2023-09-20T22:24:51Z | 2023-09-20T22:25:16Z | OWNER | The |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Move `permissions`, `allow` blocks, canned queries and more out of `metadata.yaml` and into `datasette.yaml` 1901768721 | |
1728503623 | https://github.com/simonw/datasette/pull/2191#issuecomment-1728503623 | https://api.github.com/repos/simonw/datasette/issues/2191 | IC_kwDOBm6k_c5nBt9H | simonw 9599 | 2023-09-20T22:23:33Z | 2023-09-20T22:24:10Z | OWNER | This is one of the most interesting illustrative examples in the new code: Interesting to note that it now has canned queries in it, which include this bit: It looks like metadata, but in this case it's configuration. That blur between metadata and configuration at the canned query level still feels a little bit odd to me, but I still think we're going in the right direction with it. Also interesting, from that same file: There are a few things in that metadata block that are arguably configuration, not metadata - for example: I think Most of that stuff is arguably configuration too, with the exception of the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Move `permissions`, `allow` blocks, canned queries and more out of `metadata.yaml` and into `datasette.yaml` 1901768721 | |
1728498221 | https://github.com/simonw/datasette/pull/2191#issuecomment-1728498221 | https://api.github.com/repos/simonw/datasette/issues/2191 | IC_kwDOBm6k_c5nBsot | simonw 9599 | 2023-09-20T22:17:26Z | 2023-09-20T22:17:26Z | OWNER | I tested this locally for permissions like this.
http://127.0.0.1:8001/-/metadata returned |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Move `permissions`, `allow` blocks, canned queries and more out of `metadata.yaml` and into `datasette.yaml` 1901768721 | |
1728192688 | https://github.com/simonw/datasette/issues/2189#issuecomment-1728192688 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5nAiCw | yozlet 173848 | 2023-09-20T17:53:31Z | 2023-09-20T17:53:31Z | NONE |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1726754119 | https://github.com/simonw/datasette/pull/2192#issuecomment-1726754119 | https://api.github.com/repos/simonw/datasette/issues/2192 | IC_kwDOBm6k_c5m7C1H | codecov[bot] 22429695 | 2023-09-20T01:35:45Z | 2023-09-20T01:35:45Z | NONE | Codecov ReportPatch coverage:
Additional details and impacted files```diff @@ Coverage Diff @@ ## main #2192 +/- ## ========================================== + Coverage 92.69% 92.72% +0.02% ========================================== Files 40 40 Lines 6039 6036 -3 ========================================== - Hits 5598 5597 -1 + Misses 441 439 -2 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/views/table.py](https://app.codecov.io/gh/simonw/datasette/pull/2192?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3ZpZXdzL3RhYmxlLnB5) | `96.31% <100.00%> (+0.51%)` | :arrow_up: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2192/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison):umbrella: View full report in Codecov by Sentry. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stop using parallel SQL queries for tables 1903932086 | |
1726749355 | https://github.com/simonw/datasette/issues/2189#issuecomment-1726749355 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5m7Bqr | simonw 9599 | 2023-09-20T01:28:16Z | 2023-09-20T01:28:16Z | OWNER | Added a note to that example in the documentation: https://github.com/simonw/datasette/blob/4e6a34179eaedec44c1263275d7592fd83d7e2ac/docs/internals.rst?plain=1#L1320 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1719468727 | https://github.com/simonw/datasette/pull/2185#issuecomment-1719468727 | https://api.github.com/repos/simonw/datasette/issues/2185 | IC_kwDOBm6k_c5mfQK3 | codecov[bot] 22429695 | 2023-09-14T13:36:07Z | 2023-09-19T13:42:35Z | NONE | Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files```diff @@ Coverage Diff @@ ## main #2185 +/- ## ========================================== - Coverage 92.69% 92.66% -0.04% ========================================== Files 40 40 Lines 6039 6039 ========================================== - Hits 5598 5596 -2 - Misses 441 443 +2 ``` [see 1 file with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2185/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison):umbrella: View full report in Codecov by Sentry. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump the python-packages group with 3 updates 1896578249 | |
1724325068 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724325068 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxxzM | simonw 9599 | 2023-09-18T20:29:41Z | 2023-09-18T20:29:41Z | OWNER | The one other thing affected by this change is this documentation, which suggests a not-actually-safe pattern: https://github.com/simonw/datasette/blob/6ed7908580fa2ba9297c3225d85c56f8b08b9937/docs/internals.rst#L1292-L1321 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724317367 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724317367 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxv63 | simonw 9599 | 2023-09-18T20:25:44Z | 2023-09-18T20:25:44Z | OWNER | My current hunch is that SQLite gets unhappy if multiple threads access the same underlying C object - which sometimes happens with in-memory connections and Datasette presumably because they are faster than file-backed databases. I'm going to remove the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724315591 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724315591 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxvfH | simonw 9599 | 2023-09-18T20:24:30Z | 2023-09-18T20:24:30Z | OWNER | Using SQLite In Multi-Threaded Applications That indicates that there's a SQLite option for "Serialized" mode where it's safe to access anything SQLite provides from multiple threads, but as far as I can tell Python doesn't give you an option to turn that mode on or off for a connection - you can read On my Mac |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724305169 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724305169 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxs8R | simonw 9599 | 2023-09-18T20:16:22Z | 2023-09-18T20:16:36Z | OWNER | Looking again at this code:
Python docs at https://docs.python.org/3/library/sqlite3.html
I think I'm playing with fire by allowing multiple threads to access the same connection without doing my own serialization of those requests. I do do that using the write connection - and in this particular case the bug isn't coming from write queries, it's coming from read queries - but perhaps SQLite has issues with threading for reads, too. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724298817 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724298817 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxrZB | simonw 9599 | 2023-09-18T20:11:26Z | 2023-09-18T20:11:26Z | OWNER | Now that I've confirmed that parallel query execution of the kind introduced in https://github.com/simonw/datasette/commit/942411ef946e9a34a2094944d3423cddad27efd3 can cause hangs (presumably some kind of locking issue) against in-memory databases, some options:
The parallel execution work is something I was playing with last year in the hope of speeding up Datasette pages like the table page which need to execute a bunch of queries - one for each facet, plus one for each column to see if it should be suggested as a facet. I wrote about this at the time here: https://simonwillison.net/2022/May/6/weeknotes/ My hope was that despite Python's GIL this optimization would still help, because the SQLite C module releases the GIL once it gets to SQLite. But... that didn't hold up. It looked like enough work was happening in Python land with the GIL that the optimization didn't improve things. Running the ... which it now has! But it will still be a year or two before it fully lands: https://discuss.python.org/t/a-steering-council-notice-about-pep-703-making-the-global-interpreter-lock-optional-in-cpython/30474 So I'm not particularly concerned about dropping the parallel execution. If I do drop it though do I leave the potentially complex code in that relates to it? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724281824 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724281824 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxnPg | simonw 9599 | 2023-09-18T19:58:06Z | 2023-09-18T19:58:06Z | OWNER | I also confirmed that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724278386 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724278386 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxmZy | simonw 9599 | 2023-09-18T19:55:32Z | 2023-09-18T19:55:32Z | OWNER | OK it looks like it found it! ``` 942411ef946e9a34a2094944d3423cddad27efd3 is the first bad commit commit Author: Simon Willison swillison@gmail.com Date: Tue Apr 26 15:48:56 2022 -0700
datasette/views/table.py | 93 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 67 insertions(+), 26 deletions(-) bisect found first bad commit ``` https://github.com/simonw/datasette/commit/942411ef946e9a34a2094944d3423cddad27efd3 does look like the cause of this problem. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724276917 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724276917 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxmC1 | simonw 9599 | 2023-09-18T19:54:23Z | 2023-09-18T19:54:23Z | OWNER | Turned out I wasn't running the FIxed that with Now I'm seeing some passes, which look like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724257290 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724257290 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxhQK | simonw 9599 | 2023-09-18T19:39:27Z | 2023-09-18T19:44:26Z | OWNER | I'm now trying this test script: ```bash !/bin/bashport=8064 Start datasette server in the background and get its PIDdatasette pottery2.db -p $port & server_pid=$! Wait for a moment to ensure the server has time to start upsleep 2 Initialize counters and parametersretry_count=0 max_retries=3 success_count=0 path="/airtable_refs/airtable_refs" Function to run curl with a timeoutfunction test_curl { # Run the curl command with a timeout of 3 seconds timeout 3s curl -s "http://localhost:${port}${path}" > /dev/null if [ $? -eq 0 ]; then # Curl was successful ((success_count++)) fi } Try three parallel curl requestswhile [[ $retry_count -lt $max_retries ]]; do # Reset the success counter success_count=0
done Kill the datasette serverecho "Killing datasette server with PID $server_pid" kill -9 $server_pid sleep 2 Print resultif [[ $success_count -eq 3 ]]; then
echo "All three curls succeeded."
exit 0
else
echo "Error: Not all curls succeeded after $retry_count attempts."
exit 1
fi
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724263390 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724263390 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxive | simonw 9599 | 2023-09-18T19:44:03Z | 2023-09-18T19:44:03Z | OWNER | I knocked it down to 1 retry just to see what happened. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724259229 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724259229 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxhud | simonw 9599 | 2023-09-18T19:40:56Z | 2023-09-18T19:40:56Z | OWNER | I tried it with a path of |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724258279 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724258279 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxhfn | simonw 9599 | 2023-09-18T19:40:13Z | 2023-09-18T19:40:13Z | OWNER | Output while it is running looks like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724169693 | https://github.com/simonw/datasette/pull/2190#issuecomment-1724169693 | https://api.github.com/repos/simonw/datasette/issues/2190 | IC_kwDOBm6k_c5mxL3d | codecov[bot] 22429695 | 2023-09-18T18:39:19Z | 2023-09-18T18:39:19Z | NONE | Codecov ReportPatch coverage:
Additional details and impacted files```diff @@ Coverage Diff @@ ## main #2190 +/- ## ========================================== - Coverage 92.69% 92.67% -0.03% ========================================== Files 40 40 Lines 6039 6044 +5 ========================================== + Hits 5598 5601 +3 - Misses 441 443 +2 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2190?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/app.py](https://app.codecov.io/gh/simonw/datasette/pull/2190?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.19% <100.00%> (ø)` | | | [datasette/cli.py](https://app.codecov.io/gh/simonw/datasette/pull/2190?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `82.53% <100.00%> (ø)` | | | [datasette/utils/\_\_init\_\_.py](https://app.codecov.io/gh/simonw/datasette/pull/2190?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL3V0aWxzL19faW5pdF9fLnB5) | `94.83% <100.00%> (+0.03%)` | :arrow_up: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2190/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison):umbrella: View full report in Codecov by Sentry. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Raise an exception if a "plugins" block exists in metadata.json 1901483874 | |
1724159882 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724159882 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxJeK | simonw 9599 | 2023-09-18T18:32:29Z | 2023-09-18T18:32:29Z | OWNER | This worked, including on macOS even though GPT-4 thought !/bin/bashRun the command with a timeout of 5 secondstimeout 5s datasette pottery2.db -p 8045 --get /airtable_refs/airtable_refs Check the exit code from timeoutif [ $? -eq 124 ]; then echo "Error: Command timed out after 5 seconds." exit 1 fi ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724157182 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724157182 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mxIz- | simonw 9599 | 2023-09-18T18:30:30Z | 2023-09-18T18:30:30Z | OWNER | OK, I can trigger the bug like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724089666 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724089666 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mw4VC | simonw 9599 | 2023-09-18T17:49:24Z | 2023-09-18T17:49:24Z | OWNER | I switched that particular implementation to using an on-disk database instead of an in-memory database and could no longer recreate the bug. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724084199 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724084199 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mw2_n | simonw 9599 | 2023-09-18T17:47:01Z | 2023-09-18T17:47:01Z | OWNER | I managed to trigger it by loading |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724083324 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724083324 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mw2x8 | simonw 9599 | 2023-09-18T17:46:21Z | 2023-09-18T17:46:21Z | OWNER | Sometimes it takes a few clicks for the bug to occur, but it does seem to always be within the in-memory database. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724081909 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724081909 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mw2b1 | simonw 9599 | 2023-09-18T17:45:27Z | 2023-09-18T17:45:27Z | OWNER | Maybe it's not related to faceting - I just got it on a hit to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724072390 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724072390 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mw0HG | simonw 9599 | 2023-09-18T17:39:06Z | 2023-09-18T17:39:06Z | OWNER | Landing a version of that test anyway. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724064440 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724064440 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mwyK4 | simonw 9599 | 2023-09-18T17:36:00Z | 2023-09-18T17:36:00Z | OWNER | I wrote this test, but it passes:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724055823 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724055823 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mwwEP | simonw 9599 | 2023-09-18T17:31:10Z | 2023-09-18T17:31:10Z | OWNER | That line was added in https://github.com/simonw/datasette/commit/942411ef946e9a34a2094944d3423cddad27efd3 which first shipped in 0.62a0. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724051886 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724051886 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mwvGu | simonw 9599 | 2023-09-18T17:28:20Z | 2023-09-18T17:30:30Z | OWNER | The bug exhibits when I try to add a facet. I think it's caused by the parallel query execution I added to facets at some point. http://127.0.0.1:8045/airtable_refs/airtable_refs - no error http://127.0.0.1:8045/airtable_refs/airtable_refs?_facet=table_name#facet-table_name - hangs the server Crucial line in the traceback:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724049538 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724049538 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mwuiC | simonw 9599 | 2023-09-18T17:26:44Z | 2023-09-18T17:26:44Z | OWNER | Just managed to get this exception trace:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724048314 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724048314 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mwuO6 | simonw 9599 | 2023-09-18T17:25:55Z | 2023-09-18T17:25:55Z | OWNER | The good news is that this bug is currently unlikely to affect most users since named in-memory databases (created using |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1724045748 | https://github.com/simonw/datasette/issues/2189#issuecomment-1724045748 | https://api.github.com/repos/simonw/datasette/issues/2189 | IC_kwDOBm6k_c5mwtm0 | simonw 9599 | 2023-09-18T17:24:07Z | 2023-09-18T17:24:07Z | OWNER | I need reliable steps to reproduce, then I can bisect and figure out which exact version of Datasette introduced the problem. I have a hunch that it relates to changes made to the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Server hang on parallel execution of queries to named in-memory databases 1901416155 | |
1723362847 | https://github.com/simonw/datasette/issues/2123#issuecomment-1723362847 | https://api.github.com/repos/simonw/datasette/issues/2123 | IC_kwDOBm6k_c5muG4f | hueyy 6523121 | 2023-09-18T13:02:46Z | 2023-09-18T13:02:46Z | NONE | Can confirm that this bug can be reproduced as follows:
which produces the following output:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
datasette serve when invoked with --reload interprets the serve command as a file 1825007061 | |
1722943484 | https://github.com/simonw/datasette/pull/2052#issuecomment-1722943484 | https://api.github.com/repos/simonw/datasette/issues/2052 | IC_kwDOBm6k_c5msgf8 | 20after4 30934 | 2023-09-18T08:14:47Z | 2023-09-18T08:14:47Z | NONE | This is such a well thought out contribution. I don't think I've seen such a thoroughly considered PR on any project in recent memory. |
{ "total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 1, "rocket": 0, "eyes": 0 } |
feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 1651082214 | |
1722848454 | https://github.com/simonw/datasette/issues/2188#issuecomment-1722848454 | https://api.github.com/repos/simonw/datasette/issues/2188 | IC_kwDOBm6k_c5msJTG | asg017 15178711 | 2023-09-18T06:58:53Z | 2023-09-18T06:58:53Z | CONTRIBUTOR | Thinking about this more, here a list of things I imagine a "compile-to-sql" plugin would want to do:
1) and 2) would be difficult to do with current plugin hooks, unless we add the concept of "slots" and get the JS plugin support in. 3) could maybe be done with the So it really only sounds like extending the SQL editor will be the hard part. In #2094 I want to add JavaScript plugin hooks for extending the SQL editor, which may work here. If I get the time/motivation, I might try out a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin Hooks for "compile to SQL" languages 1900026059 | |
1722845490 | https://github.com/simonw/datasette/issues/1191#issuecomment-1722845490 | https://api.github.com/repos/simonw/datasette/issues/1191 | IC_kwDOBm6k_c5msIky | asg017 15178711 | 2023-09-18T06:55:52Z | 2023-09-18T06:55:52Z | CONTRIBUTOR | One note here: this feature could be called "slots", similar to Layout Slots in Vitepress. In Vitepress, you can add custom components/widget/gadgets into determined named "slots", like so:
Would be great to do in both Python and Javascript, with the upcoming JavaScript API #2052. In |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability for plugins to collaborate when adding extra HTML to blocks in default templates 787098345 | |
1722662413 | https://github.com/simonw/datasette/issues/2188#issuecomment-1722662413 | https://api.github.com/repos/simonw/datasette/issues/2188 | IC_kwDOBm6k_c5mrb4N | simonw 9599 | 2023-09-18T02:01:34Z | 2023-09-18T02:01:34Z | OWNER | I'm not interested in these in Datasette core itself, but I think they have a ton of potential for plugins. I wonder what the best way to handle that would be? Right now it's possible to write a plugin that adds extra routes, so someone could build a If this could return JSON, they could add JavaScript to the Something that could make that more ergonomic might be the plugin hook that allows plugins to add extra HTML to different core database pages - e.g. adding a "Query this database using PRQL" button or link or even a full form at the top of that database page. That's this issue here:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin Hooks for "compile to SQL" languages 1900026059 | |
1722323967 | https://github.com/simonw/datasette/issues/2057#issuecomment-1722323967 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5mqJP_ | simonw 9599 | 2023-09-16T21:54:33Z | 2023-09-16T21:54:33Z | OWNER | Just found this migration guide: https://importlib-metadata.readthedocs.io/en/latest/migration.html |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1722266942 | https://github.com/simonw/datasette/issues/2057#issuecomment-1722266942 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5mp7U- | simonw 9599 | 2023-09-16T16:38:27Z | 2023-09-16T16:38:27Z | OWNER | The
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1722266513 | https://github.com/simonw/datasette/issues/2057#issuecomment-1722266513 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5mp7OR | simonw 9599 | 2023-09-16T16:36:09Z | 2023-09-16T16:36:09Z | OWNER | Now I need to switch out |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1722265848 | https://github.com/simonw/datasette/issues/2057#issuecomment-1722265848 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5mp7D4 | simonw 9599 | 2023-09-16T16:32:42Z | 2023-09-16T16:32:42Z | OWNER | Here's the exception it uses: ```pycon
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1722258980 | https://github.com/simonw/datasette/issues/2057#issuecomment-1722258980 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5mp5Yk | simonw 9599 | 2023-09-16T15:56:45Z | 2023-09-16T15:56:45Z | OWNER | Weird, I still can't get the warning to show even with this: ```python @pytest.mark.asyncio async def test_plugin_is_installed(): datasette = Datasette(memory=True)
``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1722257328 | https://github.com/simonw/datasette/issues/2057#issuecomment-1722257328 | https://api.github.com/repos/simonw/datasette/issues/2057 | IC_kwDOBm6k_c5mp4-w | simonw 9599 | 2023-09-16T15:47:32Z | 2023-09-16T15:47:32Z | OWNER | Frustrating that this warning doesn't show up in the Datasette test suite itself. It shows up in plugin test suites that run this test:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
DeprecationWarning: pkg_resources is deprecated as an API 1662951875 | |
1721742055 | https://github.com/simonw/datasette/issues/2186#issuecomment-1721742055 | https://api.github.com/repos/simonw/datasette/issues/2186 | IC_kwDOBm6k_c5mn7Ln | simonw 9599 | 2023-09-15T19:27:59Z | 2023-09-15T19:27:59Z | OWNER | This feels like it might be quite a nice pattern generally - providing optional arguments to plugins and views that can be |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Mechanism for register_output_renderer hooks to access full count 1898927976 | |
1721740872 | https://github.com/simonw/datasette/issues/2186#issuecomment-1721740872 | https://api.github.com/repos/simonw/datasette/issues/2186 | IC_kwDOBm6k_c5mn65I | simonw 9599 | 2023-09-15T19:26:51Z | 2023-09-15T19:27:19Z | OWNER | Here's where it's called at the moment: https://github.com/simonw/datasette/blob/16f0b6d8222d06682a31b904d0a402c391ae1c1c/datasette/views/base.py#L297-L313 And the docs: https://github.com/simonw/datasette/blob/1.0a6/docs/plugin_hooks.rst#register-output-renderer-datasette I'm tempted to add a
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Mechanism for register_output_renderer hooks to access full count 1898927976 | |
1719451803 | https://github.com/simonw/datasette/pull/2182#issuecomment-1719451803 | https://api.github.com/repos/simonw/datasette/issues/2182 | IC_kwDOBm6k_c5mfMCb | dependabot[bot] 49699333 | 2023-09-14T13:27:26Z | 2023-09-14T13:27:26Z | CONTRIBUTOR | Looks like these dependencies are updatable in another way, so this is no longer needed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump the python-packages group with 2 updates 1890593563 | |
1718316733 | https://github.com/simonw/datasette/pull/2183#issuecomment-1718316733 | https://api.github.com/repos/simonw/datasette/issues/2183 | IC_kwDOBm6k_c5ma269 | simonw 9599 | 2023-09-13T21:05:36Z | 2023-09-13T21:05:36Z | OWNER | I'm going to land this and make any further documentation tweaks on |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`datasette.yaml` plugin support 1891212159 | |
1714544153 | https://github.com/simonw/datasette/pull/2183#issuecomment-1714544153 | https://api.github.com/repos/simonw/datasette/issues/2183 | IC_kwDOBm6k_c5mMd4Z | codecov[bot] 22429695 | 2023-09-11T20:37:52Z | 2023-09-13T20:58:51Z | NONE | Codecov ReportPatch coverage:
Additional details and impacted files```diff @@ Coverage Diff @@ ## main #2183 +/- ## ========================================== - Coverage 92.69% 92.66% -0.04% ========================================== Files 40 40 Lines 6025 6039 +14 ========================================== + Hits 5585 5596 +11 - Misses 440 443 +3 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2183?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/app.py](https://app.codecov.io/gh/simonw/datasette/pull/2183?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.19% <95.00%> (-0.24%)` | :arrow_down: |:umbrella: View full report in Codecov by Sentry. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`datasette.yaml` plugin support 1891212159 | |
1716801971 | https://github.com/simonw/datasette/pull/2183#issuecomment-1716801971 | https://api.github.com/repos/simonw/datasette/issues/2183 | IC_kwDOBm6k_c5mVFGz | asg017 15178711 | 2023-09-13T01:34:01Z | 2023-09-13T01:34:01Z | CONTRIBUTOR | @simonw docs are finished, this is ready for review! One thing: I added "Configuration" as a top-level item in the documentation site, at the very bottom. Not sure if this is the best, maybe it can be named "datasette.yaml Configuration" or something similar? Mostly because "Configuration" by itself can mean many things, but adding "datasette.yaml" would make it pretty clear it's about that specific file, and is easier to scan. I'd also be fine with using "datasette.yaml" instead of "datasette.json", since writing in YAML is much more forgiving (and advanced users will know JSON is also supported) Also, maybe this is a chance to consolidate the docs a bit? I think "Settings", "Configuration", "Metadata", and "Authentication and permissions" should possibly be under the same section. Maybe even consolidate the different Plugin pages that exist? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`datasette.yaml` plugin support 1891212159 | |
1714920708 | https://github.com/simonw/sqlite-utils/issues/594#issuecomment-1714920708 | https://api.github.com/repos/simonw/sqlite-utils/issues/594 | IC_kwDOCGYnMM5mN50E | simonw 9599 | 2023-09-12T03:51:13Z | 2023-09-12T03:51:13Z | OWNER | Changing this without breaking backwards compatibility (and forcing a 4.0 release) will be tricky, because I could swap it out for a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Represent compound foreign keys in table.foreign_keys output 1891614971 | |
1714919806 | https://github.com/simonw/sqlite-utils/issues/594#issuecomment-1714919806 | https://api.github.com/repos/simonw/sqlite-utils/issues/594 | IC_kwDOCGYnMM5mN5l- | simonw 9599 | 2023-09-12T03:49:41Z | 2023-09-12T03:49:41Z | OWNER | Digging in a bit more: ```pycon
Right now I ignore those columns entirely: https://github.com/simonw/sqlite-utils/blob/622c3a5a7dd53a09c029e2af40c2643fe7579340/sqlite_utils/db.py#L1523-L1540 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Represent compound foreign keys in table.foreign_keys output 1891614971 | |
1714699724 | https://github.com/simonw/datasette/pull/2183#issuecomment-1714699724 | https://api.github.com/repos/simonw/datasette/issues/2183 | IC_kwDOBm6k_c5mND3M | simonw 9599 | 2023-09-11T23:01:36Z | 2023-09-11T23:02:30Z | OWNER | On thinking about this further, I'm fine releasing it as another alpha provided it causes Datasette to error loudly with an explanatory message if you attempt to load Something like this:
For added usability points, let's have it suggest |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`datasette.yaml` plugin support 1891212159 | |
1712897194 | https://github.com/simonw/datasette/issues/2168#issuecomment-1712897194 | https://api.github.com/repos/simonw/datasette/issues/2168 | IC_kwDOBm6k_c5mGLyq | simonw 9599 | 2023-09-10T17:54:07Z | 2023-09-10T17:54:07Z | OWNER | This looks relevant: https://pluggy.readthedocs.io/en/stable/#wrappers
This could be the perfect mechanism for implementing this hook, although I still need to figure out how it interacts with streaming. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Consider a request/response wrapping hook slightly higher level than asgi_wrapper() 1876353656 | |
1712895580 | https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1712895580 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | IC_kwDOCGYnMM5mGLZc | simonw 9599 | 2023-09-10T17:46:41Z | 2023-09-10T17:46:41Z | OWNER | In working on this I learned that https://www.sqlite.org/lang_vacuum.html#how_vacuum_works
So this fix wasn't as valuable as I thought. I need to move away from ever assuming that a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`table.transform()` should preserve `rowid` values 1886771493 | |
1712895084 | https://github.com/simonw/datasette/issues/2180#issuecomment-1712895084 | https://api.github.com/repos/simonw/datasette/issues/2180 | IC_kwDOBm6k_c5mGLRs | simonw 9599 | 2023-09-10T17:44:14Z | 2023-09-10T17:44:14Z | OWNER | Used by |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook: `actors_from_ids()` 1886791100 | |
1710939868 | https://github.com/simonw/sqlite-utils/pull/593#issuecomment-1710939868 | https://api.github.com/repos/simonw/sqlite-utils/issues/593 | IC_kwDOCGYnMM5l-t7c | codecov[bot] 22429695 | 2023-09-08T01:03:40Z | 2023-09-09T00:44:52Z | NONE | Codecov ReportPatch coverage:
Additional details and impacted files```diff @@ Coverage Diff @@ ## main #593 +/- ## ======================================= Coverage 95.77% 95.77% ======================================= Files 8 8 Lines 2837 2840 +3 ======================================= + Hits 2717 2720 +3 Misses 120 120 ``` | [Files Changed](https://app.codecov.io/gh/simonw/sqlite-utils/pull/593?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [sqlite\_utils/db.py](https://app.codecov.io/gh/simonw/sqlite-utils/pull/593?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-c3FsaXRlX3V0aWxzL2RiLnB5) | `97.22% <100.00%> (+<0.01%)` | :arrow_up: |:umbrella: View full report in Codecov by Sentry. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.transform() now preserves rowid values, refs #592 1886783150 | |
1711057080 | https://github.com/simonw/datasette/issues/2180#issuecomment-1711057080 | https://api.github.com/repos/simonw/datasette/issues/2180 | IC_kwDOBm6k_c5l_Ki4 | simonw 9599 | 2023-09-08T04:28:03Z | 2023-09-08T04:28:03Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook: `actors_from_ids()` 1886791100 | ||
1711054840 | https://github.com/simonw/datasette/pull/2181#issuecomment-1711054840 | https://api.github.com/repos/simonw/datasette/issues/2181 | IC_kwDOBm6k_c5l_J_4 | simonw 9599 | 2023-09-08T04:23:47Z | 2023-09-08T04:23:47Z | OWNER | I've implemented this hook once now in: - https://github.com/datasette/datasette-remote-actors And built and tested a debug tool for it in: - https://github.com/datasette/datasette-debug-actors-from-ids I'm now confident in the design of this plugin hook, I'm going to land it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
actors_from_ids plugin hook and datasette.actors_from_ids() method 1886812002 | |
1711054624 | https://github.com/simonw/datasette/issues/2180#issuecomment-1711054624 | https://api.github.com/repos/simonw/datasette/issues/2180 | IC_kwDOBm6k_c5l_J8g | simonw 9599 | 2023-09-08T04:23:23Z | 2023-09-08T04:23:23Z | OWNER | I've implemented this hook once now in: - https://github.com/datasette/datasette-remote-actors And built and tested a debug tool for it in: - https://github.com/datasette/datasette-debug-actors-from-ids I'm now confident in the design of this plugin hook, I'm going to land it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook: `actors_from_ids()` 1886791100 | |
1710972324 | https://github.com/simonw/datasette/pull/2181#issuecomment-1710972324 | https://api.github.com/repos/simonw/datasette/issues/2181 | IC_kwDOBm6k_c5l-12k | codecov[bot] 22429695 | 2023-09-08T01:58:44Z | 2023-09-08T03:43:43Z | NONE | Codecov ReportPatch coverage:
Additional details and impacted files```diff @@ Coverage Diff @@ ## main #2181 +/- ## ======================================= Coverage 92.68% 92.69% ======================================= Files 40 40 Lines 6017 6025 +8 ======================================= + Hits 5577 5585 +8 Misses 440 440 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/app.py](https://app.codecov.io/gh/simonw/datasette/pull/2181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2FwcC5weQ==) | `94.43% <100.00%> (+0.03%)` | :arrow_up: | | [datasette/hookspecs.py](https://app.codecov.io/gh/simonw/datasette/pull/2181?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2hvb2tzcGVjcy5weQ==) | `100.00% <100.00%> (ø)` | |:umbrella: View full report in Codecov by Sentry. :loudspeaker: Have feedback on the report? Share it here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
actors_from_ids plugin hook and datasette.actors_from_ids() method 1886812002 | |
1711028355 | https://github.com/simonw/datasette/issues/2180#issuecomment-1711028355 | https://api.github.com/repos/simonw/datasette/issues/2180 | IC_kwDOBm6k_c5l_DiD | simonw 9599 | 2023-09-08T03:36:02Z | 2023-09-08T03:36:02Z | OWNER | I shipped the first version of When I land this plugin in a Datasette release I need to update that repo to depend on the new alpha. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook: `actors_from_ids()` 1886791100 | |
1710969448 | https://github.com/simonw/datasette/pull/2181#issuecomment-1710969448 | https://api.github.com/repos/simonw/datasette/issues/2181 | IC_kwDOBm6k_c5l-1Jo | simonw 9599 | 2023-09-08T01:53:27Z | 2023-09-08T01:53:27Z | OWNER | Documentation preview: - https://datasette--2181.org.readthedocs.build/en/2181/internals.html#await-actors-from-ids-actor-ids - https://datasette--2181.org.readthedocs.build/en/2181/plugin_hooks.html#plugin-hook-actors-from-ids |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
actors_from_ids plugin hook and datasette.actors_from_ids() method 1886812002 | |
1710969339 | https://github.com/simonw/datasette/issues/2180#issuecomment-1710969339 | https://api.github.com/repos/simonw/datasette/issues/2180 | IC_kwDOBm6k_c5l-1H7 | simonw 9599 | 2023-09-08T01:53:16Z | 2023-09-08T01:53:16Z | OWNER | Documentation preview: - https://datasette--2181.org.readthedocs.build/en/2181/internals.html#await-actors-from-ids-actor-ids - https://datasette--2181.org.readthedocs.build/en/2181/plugin_hooks.html#plugin-hook-actors-from-ids |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook: `actors_from_ids()` 1886791100 | |
1710950671 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-1710950671 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8 | IC_kwDODFE5qs5l-wkP | iloveitaly 150855 | 2023-09-08T01:22:49Z | 2023-09-08T01:22:49Z | NONE | Makes sense, thanks for explaining! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add Gmail takeout mbox import (v2) 954546309 | |
1710947637 | https://github.com/simonw/datasette/issues/2180#issuecomment-1710947637 | https://api.github.com/repos/simonw/datasette/issues/2180 | IC_kwDOBm6k_c5l-v01 | simonw 9599 | 2023-09-08T01:17:28Z | 2023-09-08T01:17:28Z | OWNER | I think this is both a plugin hook and a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook: `actors_from_ids()` 1886791100 | |
1710935270 | https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710935270 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | IC_kwDOCGYnMM5l-szm | simonw 9599 | 2023-09-08T00:55:30Z | 2023-09-08T00:55:30Z | OWNER | Yes! That recreated the bug: ```
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`table.transform()` should preserve `rowid` values 1886771493 | |
1710934448 | https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710934448 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | IC_kwDOCGYnMM5l-smw | simonw 9599 | 2023-09-08T00:54:03Z | 2023-09-08T00:54:03Z | OWNER | Oh! Maybe the row ID preservation here is a coincidence because the tables are created from scratch and count 1, 2, 3. If I delete a row from the table and then insert some more - breaking the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`table.transform()` should preserve `rowid` values 1886771493 | |
1710933716 | https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710933716 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | IC_kwDOCGYnMM5l-sbU | simonw 9599 | 2023-09-08T00:52:42Z | 2023-09-08T00:52:42Z | OWNER | I just noticed that the table where I encountered this bug wasn't actually a The reason the But I tried this test and it passed, too:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`table.transform()` should preserve `rowid` values 1886771493 | |
1710931605 | https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710931605 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | IC_kwDOCGYnMM5l-r6V | simonw 9599 | 2023-09-08T00:49:02Z | 2023-09-08T00:49:02Z | OWNER | I tried bumping that up to 10,000 rows instead of just 3 but the test still passed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`table.transform()` should preserve `rowid` values 1886771493 | |
1710930934 | https://github.com/simonw/sqlite-utils/issues/592#issuecomment-1710930934 | https://api.github.com/repos/simonw/sqlite-utils/issues/592 | IC_kwDOCGYnMM5l-rv2 | simonw 9599 | 2023-09-08T00:47:57Z | 2023-09-08T00:47:57Z | OWNER | That's odd, I wrote a test for this just now and it passes already:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`table.transform()` should preserve `rowid` values 1886771493 | |
1708699926 | https://github.com/simonw/datasette/pull/2174#issuecomment-1708699926 | https://api.github.com/repos/simonw/datasette/issues/2174 | IC_kwDOBm6k_c5l2LEW | codecov[bot] 22429695 | 2023-09-06T16:18:13Z | 2023-09-08T00:46:13Z | NONE | Codecov ReportPatch and project coverage have no change.
Additional details and impacted files```diff @@ Coverage Diff @@ ## main #2174 +/- ## ======================================= Coverage 92.68% 92.68% ======================================= Files 40 40 Lines 6012 6017 +5 ======================================= + Hits 5572 5577 +5 Misses 440 440 ``` | [Files Changed](https://app.codecov.io/gh/simonw/datasette/pull/2174?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) | Coverage Δ | | |---|---|---| | [datasette/cli.py](https://app.codecov.io/gh/simonw/datasette/pull/2174?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison#diff-ZGF0YXNldHRlL2NsaS5weQ==) | `82.53% <ø> (ø)` | | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/simonw/datasette/pull/2174/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison):umbrella: View full report in Codecov by Sentry. :loudspeaker: Have feedback on the report? Share it here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use $DATASETTE_INTERNAL in absence of --internal 1884330740 | |
1708727204 | https://github.com/simonw/datasette/pull/2174#issuecomment-1708727204 | https://api.github.com/repos/simonw/datasette/issues/2174 | IC_kwDOBm6k_c5l2Ruk | simonw 9599 | 2023-09-06T16:35:39Z | 2023-09-07T23:45:42Z | OWNER | We can use this here: https://click.palletsprojects.com/en/8.1.x/options/#values-from-environment-variables
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use $DATASETTE_INTERNAL in absence of --internal 1884330740 | |
1710879239 | https://github.com/simonw/datasette/issues/2178#issuecomment-1710879239 | https://api.github.com/repos/simonw/datasette/issues/2178 | IC_kwDOBm6k_c5l-fIH | simonw 9599 | 2023-09-07T23:20:32Z | 2023-09-07T23:20:32Z | OWNER | To test that locally, use this YAML instead:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Don't show foreign key links to tables the user cannot access 1886350562 | |
1710878391 | https://github.com/simonw/datasette/issues/2178#issuecomment-1710878391 | https://api.github.com/repos/simonw/datasette/issues/2178 | IC_kwDOBm6k_c5l-e63 | simonw 9599 | 2023-09-07T23:19:05Z | 2023-09-07T23:19:05Z | OWNER | This fix didn't work on Datasette Cloud. I used Only checking |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Don't show foreign key links to tables the user cannot access 1886350562 | |
1710871095 | https://github.com/simonw/datasette/issues/2178#issuecomment-1710871095 | https://api.github.com/repos/simonw/datasette/issues/2178 | IC_kwDOBm6k_c5l-dI3 | simonw 9599 | 2023-09-07T23:07:16Z | 2023-09-07T23:07:16Z | OWNER | I ran this:
Against this YAML:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Don't show foreign key links to tables the user cannot access 1886350562 | |
1710567329 | https://github.com/simonw/datasette/issues/2178#issuecomment-1710567329 | https://api.github.com/repos/simonw/datasette/issues/2178 | IC_kwDOBm6k_c5l9S-h | simonw 9599 | 2023-09-07T17:59:59Z | 2023-09-07T17:59:59Z | OWNER | Should I put the permission check in that undocumented |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Don't show foreign key links to tables the user cannot access 1886350562 | |
1710565268 | https://github.com/simonw/datasette/issues/2178#issuecomment-1710565268 | https://api.github.com/repos/simonw/datasette/issues/2178 | IC_kwDOBm6k_c5l9SeU | simonw 9599 | 2023-09-07T17:58:04Z | 2023-09-07T17:59:06Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Don't show foreign key links to tables the user cannot access 1886350562 | ||
1710380941 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-1710380941 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8 | IC_kwDODFE5qs5l8leN | maxhawkins 28565 | 2023-09-07T15:39:59Z | 2023-09-07T15:39:59Z | NONE |
Mailbox parses the entire mbox into memory. Using the lower level library lets us stream the emails in one at a time to support larger archives. Both libraries are in the stdlib. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add Gmail takeout mbox import (v2) 954546309 | |
1709373304 | https://github.com/simonw/datasette/issues/1479#issuecomment-1709373304 | https://api.github.com/repos/simonw/datasette/issues/1479 | IC_kwDOBm6k_c5l4vd4 | hcarter333 363004 | 2023-09-07T02:14:15Z | 2023-09-07T02:14:15Z | NONE | I ran into the same issue on Windows using
@Rik-de-Kort your fix worked perfectly! Thanks! I can always go back and delete the temp directories myself :) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Win32 "used by another process" error with datasette publish 1010112818 | |
1708945716 | https://github.com/dogsheep/google-takeout-to-sqlite/pull/8#issuecomment-1708945716 | https://api.github.com/repos/dogsheep/google-takeout-to-sqlite/issues/8 | IC_kwDODFE5qs5l3HE0 | iloveitaly 150855 | 2023-09-06T19:12:33Z | 2023-09-06T19:12:33Z | NONE | @maxhawkins curious why you didn't use the stdlib |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add Gmail takeout mbox import (v2) 954546309 | |
1708777964 | https://github.com/simonw/datasette/issues/2177#issuecomment-1708777964 | https://api.github.com/repos/simonw/datasette/issues/2177 | IC_kwDOBm6k_c5l2eHs | simonw 9599 | 2023-09-06T17:04:30Z | 2023-09-06T17:04:30Z | OWNER | Here's the thinking: setting the You might run more than one instance at once (I often have 4 or 5 going). This would currently break, because they would over-write each other's catalog tables: The breaking wouldn't be obvious because the catalog tables aren't used by any features yet, but it's still bad. This convinced us that actually we should move those I think we move them to an in-memory So each instance of Datasette gets its own truly private An interesting thing that came up about a shared |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Move schema tables from _internal to _catalog 1884408624 | |
1708728926 | https://github.com/simonw/datasette/pull/2174#issuecomment-1708728926 | https://api.github.com/repos/simonw/datasette/issues/2174 | IC_kwDOBm6k_c5l2SJe | simonw 9599 | 2023-09-06T16:36:56Z | 2023-09-06T16:36:56Z | OWNER |
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use $DATASETTE_INTERNAL in absence of --internal 1884330740 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
user >30