issue_comments
8,358 rows where author_association = "OWNER" and user = 9599 sorted by updated_at descending
This data as json, CSV (advanced)
These facets timed out: issue
id | html_url | issue_url | node_id | user | created_at | updated_at ▲ | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1133254599 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133254599 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5DjBfH | simonw 9599 | 2022-05-20T19:33:08Z | 2022-05-20T19:33:08Z | OWNER | Actually maybe I don't? I just noticed that on other pages on https://docs.datasette.io/en/stable/installation.html the only way to get back to that useful table of context / index page at https://docs.datasette.io/en/stable/index.html is by clicking the tiny house icon. Can I do better or should I have the logo do that? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133252598 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133252598 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5DjA_2 | simonw 9599 | 2022-05-20T19:31:30Z | 2022-05-20T19:31:30Z | OWNER | I'd also like to bring back this stable / latest / version indicator: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133250151 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133250151 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5DjAZn | simonw 9599 | 2022-05-20T19:29:37Z | 2022-05-20T19:29:37Z | OWNER | I want the Datasette logo in the sidebar to link to https://datasette.io/ Looks like I can do that by dropping in my own |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133246791 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133246791 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5Di_lH | simonw 9599 | 2022-05-20T19:26:49Z | 2022-05-20T19:26:49Z | OWNER | Putting this in the
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133242063 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133242063 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5Di-bP | simonw 9599 | 2022-05-20T19:22:49Z | 2022-05-20T19:22:49Z | OWNER | I have some custom CSS in this file: I tested and the The |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133232301 | https://github.com/simonw/datasette/issues/1748#issuecomment-1133232301 | https://api.github.com/repos/simonw/datasette/issues/1748 | IC_kwDOBm6k_c5Di8Ct | simonw 9599 | 2022-05-20T19:15:00Z | 2022-05-20T19:15:00Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add copy buttons next to code examples in the documentation 1243517592 | ||
1133229196 | https://github.com/simonw/datasette/issues/1747#issuecomment-1133229196 | https://api.github.com/repos/simonw/datasette/issues/1747 | IC_kwDOBm6k_c5Di7SM | simonw 9599 | 2022-05-20T19:12:30Z | 2022-05-20T19:12:30Z | OWNER | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add tutorials to the getting started guide 1243512344 | ||
1133225441 | https://github.com/simonw/datasette/issues/1748#issuecomment-1133225441 | https://api.github.com/repos/simonw/datasette/issues/1748 | IC_kwDOBm6k_c5Di6Xh | simonw 9599 | 2022-05-20T19:09:13Z | 2022-05-20T19:09:13Z | OWNER | I'm going to add this Sphinx plugin: https://github.com/executablebooks/sphinx-copybutton |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add copy buttons next to code examples in the documentation 1243517592 | |
1133222848 | https://github.com/simonw/datasette/issues/1153#issuecomment-1133222848 | https://api.github.com/repos/simonw/datasette/issues/1153 | IC_kwDOBm6k_c5Di5vA | simonw 9599 | 2022-05-20T19:07:10Z | 2022-05-20T19:07:10Z | OWNER | I could use https://github.com/pradyunsg/sphinx-inline-tabs for this - recommended by https://pradyunsg.me/furo/recommendations/ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Use YAML examples in documentation by default, not JSON 771202454 | |
1133217219 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133217219 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5Di4XD | simonw 9599 | 2022-05-20T18:58:54Z | 2022-05-20T18:58:54Z | OWNER | Need to address other customizations I've made in https://github.com/simonw/datasette/blob/0.62a0/docs/_templates/layout.html - such as Plausible analytics and some custom JavaScript. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133215684 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133215684 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5Di3_E | simonw 9599 | 2022-05-20T18:56:29Z | 2022-05-20T18:56:29Z | OWNER | One other problem: in dark mode the Datasette logo looks bad: This helps a bit:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133210942 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133210942 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5Di20- | simonw 9599 | 2022-05-20T18:49:40Z | 2022-05-20T18:49:40Z | OWNER | And for those local table of contents, do this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133210651 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133210651 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5Di2wb | simonw 9599 | 2022-05-20T18:49:11Z | 2022-05-20T18:49:11Z | OWNER | I found a workaround for the no-longer-nested left hand navigation: drop this into ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1133210032 | https://github.com/simonw/datasette/issues/1746#issuecomment-1133210032 | https://api.github.com/repos/simonw/datasette/issues/1746 | IC_kwDOBm6k_c5Di2mw | simonw 9599 | 2022-05-20T18:48:17Z | 2022-05-20T18:48:17Z | OWNER | A couple of changes I want to make. First, I don't really like the way Furo keeps the in-page titles in a separate menu on the right rather than expanding them on the left. I like this: Furo wants to do this instead: I also still want to include those inline tables of contents on the two pages that have them: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Switch documentation theme to Furo 1243498298 | |
1129251699 | https://github.com/simonw/datasette/issues/1744#issuecomment-1129251699 | https://api.github.com/repos/simonw/datasette/issues/1744 | IC_kwDOBm6k_c5DTwNz | simonw 9599 | 2022-05-17T19:44:47Z | 2022-05-17T19:46:38Z | OWNER | { "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`--nolock` feature for opening locked databases 1239008850 | ||
1129252603 | https://github.com/simonw/datasette/issues/1745#issuecomment-1129252603 | https://api.github.com/repos/simonw/datasette/issues/1745 | IC_kwDOBm6k_c5DTwb7 | simonw 9599 | 2022-05-17T19:45:51Z | 2022-05-17T19:45:51Z | OWNER | Now documented here: https://docs.datasette.io/en/latest/contributing.html#running-cog |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Documentation on running cog 1239080102 | |
1129243427 | https://github.com/simonw/datasette/issues/1744#issuecomment-1129243427 | https://api.github.com/repos/simonw/datasette/issues/1744 | IC_kwDOBm6k_c5DTuMj | simonw 9599 | 2022-05-17T19:35:02Z | 2022-05-17T19:35:02Z | OWNER | One thing to note is that the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`--nolock` feature for opening locked databases 1239008850 | |
1129241873 | https://github.com/simonw/datasette/issues/1744#issuecomment-1129241873 | https://api.github.com/repos/simonw/datasette/issues/1744 | IC_kwDOBm6k_c5DTt0R | simonw 9599 | 2022-05-17T19:33:16Z | 2022-05-17T19:33:16Z | OWNER | I'm going to skip adding a test for this - the test logic would have to be pretty convoluted to exercise it properly, and it's a pretty minor and low-risk feature in the scheme of things. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`--nolock` feature for opening locked databases 1239008850 | |
1129241283 | https://github.com/simonw/datasette/issues/1744#issuecomment-1129241283 | https://api.github.com/repos/simonw/datasette/issues/1744 | IC_kwDOBm6k_c5DTtrD | simonw 9599 | 2022-05-17T19:32:35Z | 2022-05-17T19:32:35Z | OWNER | I tried writing a test like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`--nolock` feature for opening locked databases 1239008850 | |
1129187486 | https://github.com/simonw/datasette/issues/1744#issuecomment-1129187486 | https://api.github.com/repos/simonw/datasette/issues/1744 | IC_kwDOBm6k_c5DTgie | simonw 9599 | 2022-05-17T18:28:49Z | 2022-05-17T18:28:49Z | OWNER | I think I do that with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`--nolock` feature for opening locked databases 1239008850 | |
1129185356 | https://github.com/simonw/datasette/issues/1744#issuecomment-1129185356 | https://api.github.com/repos/simonw/datasette/issues/1744 | IC_kwDOBm6k_c5DTgBM | simonw 9599 | 2022-05-17T18:26:26Z | 2022-05-17T18:26:26Z | OWNER | Not sure how to test this - I'd need to open my own lock against a database file somehow. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`--nolock` feature for opening locked databases 1239008850 | |
1129184908 | https://github.com/simonw/datasette/issues/1744#issuecomment-1129184908 | https://api.github.com/repos/simonw/datasette/issues/1744 | IC_kwDOBm6k_c5DTf6M | simonw 9599 | 2022-05-17T18:25:57Z | 2022-05-17T18:25:57Z | OWNER | I knocked out a quick prototype of this and it worked!
Here's the prototype diff: ```diff diff --git a/datasette/app.py b/datasette/app.py index b7b8437..f43700d 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -213,6 +213,7 @@ class Datasette: config_dir=None, pdb=False, crossdb=False, + nolock=False, ): assert config_dir is None or isinstance( config_dir, Path @@ -238,6 +239,7 @@ class Datasette: self.databases = collections.OrderedDict() self._refresh_schemas_lock = asyncio.Lock() self.crossdb = crossdb + self.nolock = nolock if memory or crossdb or not self.files: self.add_database(Database(self, is_memory=True), name="_memory") # memory_name is a random string so that each Datasette instance gets its own diff --git a/datasette/cli.py b/datasette/cli.py index 3c6e1b2..7e44665 100644 --- a/datasette/cli.py +++ b/datasette/cli.py @@ -452,6 +452,11 @@ def uninstall(packages, yes): is_flag=True, help="Enable cross-database joins using the /_memory database", ) +@click.option( + "--nolock", + is_flag=True, + help="Ignore locking and open locked files in read-only mode", +) @click.option( "--ssl-keyfile", help="SSL key file", @@ -486,6 +491,7 @@ def serve( open_browser, create, crossdb, + nolock, ssl_keyfile, ssl_certfile, return_instance=False, @@ -545,6 +551,7 @@ def serve( version_note=version_note, pdb=pdb, crossdb=crossdb, + nolock=nolock, )
diff --git a/datasette/database.py b/datasette/database.py index 44d3266..fa55804 100644 --- a/datasette/database.py +++ b/datasette/database.py @@ -89,6 +89,8 @@ class Database: # mode=ro or immutable=1? if self.is_mutable: qs = "?mode=ro" + if self.ds.nolock: + qs += "&nolock=1" else: qs = "?immutable=1" assert not (write and not self.is_mutable) ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`--nolock` feature for opening locked databases 1239008850 | |
1128052948 | https://github.com/simonw/datasette/issues/1742#issuecomment-1128052948 | https://api.github.com/repos/simonw/datasette/issues/1742 | IC_kwDOBm6k_c5DPLjU | simonw 9599 | 2022-05-16T19:28:31Z | 2022-05-16T19:28:31Z | OWNER | The trace mechanism is a bit gnarly - it's actually done by some ASGI middleware I wrote, so I'm pretty sure the bug is in there somewhere: https://github.com/simonw/datasette/blob/280ff372ab30df244f6c54f6f3002da57334b3d7/datasette/tracer.py#L73 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_trace=1 fails with datasette-geojson for some reason 1237586379 | |
1128033018 | https://github.com/simonw/datasette/issues/1742#issuecomment-1128033018 | https://api.github.com/repos/simonw/datasette/issues/1742 | IC_kwDOBm6k_c5DPGr6 | simonw 9599 | 2022-05-16T19:06:38Z | 2022-05-16T19:06:38Z | OWNER | The same URL with |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_trace=1 fails with datasette-geojson for some reason 1237586379 | |
1117662420 | https://github.com/simonw/datasette/issues/1739#issuecomment-1117662420 | https://api.github.com/repos/simonw/datasette/issues/1739 | IC_kwDOBm6k_c5CnizU | simonw 9599 | 2022-05-04T18:21:18Z | 2022-05-04T18:21:18Z | OWNER | That prototype is now public: https://github.com/simonw/datasette-lite |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.db downloads should be served with an ETag 1223699280 | |
1116215371 | https://github.com/simonw/datasette/issues/1739#issuecomment-1116215371 | https://api.github.com/repos/simonw/datasette/issues/1739 | IC_kwDOBm6k_c5CiBhL | simonw 9599 | 2022-05-03T15:12:16Z | 2022-05-03T15:12:16Z | OWNER | That worked - both DBs are 304 for me now on a subsequent load of the page: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.db downloads should be served with an ETag 1223699280 | |
1116183369 | https://github.com/simonw/datasette/issues/1739#issuecomment-1116183369 | https://api.github.com/repos/simonw/datasette/issues/1739 | IC_kwDOBm6k_c5Ch5tJ | simonw 9599 | 2022-05-03T14:43:14Z | 2022-05-03T14:43:14Z | OWNER | Relevant tests start here: https://github.com/simonw/datasette/blob/d60f163528f466b1127b2935c3b6869c34fd6545/tests/test_html.py#L395 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.db downloads should be served with an ETag 1223699280 | |
1116180599 | https://github.com/simonw/datasette/issues/1739#issuecomment-1116180599 | https://api.github.com/repos/simonw/datasette/issues/1739 | IC_kwDOBm6k_c5Ch5B3 | simonw 9599 | 2022-05-03T14:40:32Z | 2022-05-03T14:40:32Z | OWNER | Database downloads are served here: https://github.com/simonw/datasette/blob/d60f163528f466b1127b2935c3b6869c34fd6545/datasette/views/database.py#L186-L192 Here's I can add an |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.db downloads should be served with an ETag 1223699280 | |
1116178727 | https://github.com/simonw/datasette/issues/1739#issuecomment-1116178727 | https://api.github.com/repos/simonw/datasette/issues/1739 | IC_kwDOBm6k_c5Ch4kn | simonw 9599 | 2022-05-03T14:38:46Z | 2022-05-03T14:38:46Z | OWNER | Reminded myself how this works by reviewing Simply add a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.db downloads should be served with an ETag 1223699280 | |
1115760104 | https://github.com/simonw/datasette/issues/1739#issuecomment-1115760104 | https://api.github.com/repos/simonw/datasette/issues/1739 | IC_kwDOBm6k_c5CgSXo | simonw 9599 | 2022-05-03T05:50:19Z | 2022-05-03T05:50:19Z | OWNER | Here's how Starlette does it: https://github.com/encode/starlette/blob/830f3486537916bae6b46948ff922adc14a22b7c/starlette/staticfiles.py#L213 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
.db downloads should be served with an ETag 1223699280 | |
1115533820 | https://github.com/simonw/datasette/issues/1732#issuecomment-1115533820 | https://api.github.com/repos/simonw/datasette/issues/1732 | IC_kwDOBm6k_c5CfbH8 | simonw 9599 | 2022-05-03T01:42:25Z | 2022-05-03T01:42:25Z | OWNER | Thanks, this definitely sounds like a bug. Do you have simple steps to reproduce this? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Custom page variables aren't decoded 1221849746 | |
1115470180 | https://github.com/simonw/datasette/issues/1737#issuecomment-1115470180 | https://api.github.com/repos/simonw/datasette/issues/1737 | IC_kwDOBm6k_c5CfLlk | simonw 9599 | 2022-05-02T23:39:29Z | 2022-05-02T23:39:29Z | OWNER | Test ran in 38 seconds and passed! https://github.com/simonw/datasette/runs/6265954274?check_suite_focus=true I'm going to have it run on every commit and PR. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Automated test for Pyodide compatibility 1223459734 | |
1115468193 | https://github.com/simonw/datasette/issues/1737#issuecomment-1115468193 | https://api.github.com/repos/simonw/datasette/issues/1737 | IC_kwDOBm6k_c5CfLGh | simonw 9599 | 2022-05-02T23:35:26Z | 2022-05-02T23:35:26Z | OWNER | https://github.com/simonw/datasette/runs/6265915080?check_suite_focus=true failed but looks like it passed because I forgot to use It failed because it didn't have |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Automated test for Pyodide compatibility 1223459734 | |
1115464097 | https://github.com/simonw/datasette/issues/1737#issuecomment-1115464097 | https://api.github.com/repos/simonw/datasette/issues/1737 | IC_kwDOBm6k_c5CfKGh | simonw 9599 | 2022-05-02T23:27:40Z | 2022-05-02T23:27:40Z | OWNER | I'm going to start off by running this manually - I may run it on every commit once this is all a little bit more stable. I can base the workflow on https://github.com/simonw/scrape-hacker-news-by-domain/blob/main/.github/workflows/scrape.yml |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Automated test for Pyodide compatibility 1223459734 | |
1115462720 | https://github.com/simonw/datasette/issues/1737#issuecomment-1115462720 | https://api.github.com/repos/simonw/datasette/issues/1737 | IC_kwDOBm6k_c5CfJxA | simonw 9599 | 2022-05-02T23:25:03Z | 2022-05-02T23:25:03Z | OWNER | Here's a script that seems to work. It builds the wheel, starts a Python web server that serves the wheel, runs a test with ```bash !/bin/bashBuild the wheelpython3 -m build Find name of wheelwheel=$(basename $(ls dist/*.whl)) strip off the dist/Create a blank index pageecho ' <script src="https://cdn.jsdelivr.net/pyodide/v0.20.0/full/pyodide.js"></script>' > dist/index.html Run a server for that dist/ foldercd dist python3 -m http.server 8529 & cd .. shot-scraper javascript http://localhost:8529/ " async () => { let pyodide = await loadPyodide(); await pyodide.loadPackage(['micropip', 'ssl', 'setuptools']); let output = await pyodide.runPythonAsync(` import micropip await micropip.install('h11==0.12.0') await micropip.install('http://localhost:8529/$wheel') import ssl import setuptools from datasette.app import Datasette ds = Datasette(memory=True, settings={'num_sql_threads': 0}) (await ds.client.get('/_memory.json?sql=select+55+as+itworks&_shape=array')).text `); if (JSON.parse(output)[0].itworks != 55) { throw 'Got ' + output + ', expected itworks: 55'; } return 'Test passed!'; } " Shut down the serverpkill -f 'http.server 8529' ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Automated test for Pyodide compatibility 1223459734 | |
1115404729 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115404729 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5Ce7m5 | simonw 9599 | 2022-05-02T21:49:01Z | 2022-05-02T21:49:38Z | OWNER | That alpha release works! https://pyodide.org/en/stable/console.html ```pycon Welcome to the Pyodide terminal emulator 🐍 Python 3.10.2 (main, Apr 9 2022 20:52:01) on WebAssembly VM Type "help", "copyright", "credits" or "license" for more information.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115318417 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115318417 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5CemiR | simonw 9599 | 2022-05-02T20:13:43Z | 2022-05-02T20:13:43Z | OWNER | This is good enough to push an alpha. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115318303 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115318303 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5Cemgf | simonw 9599 | 2022-05-02T20:13:36Z | 2022-05-02T20:13:36Z | OWNER | I got a build from the ``` Welcome to the Pyodide terminal emulator 🐍 Python 3.10.2 (main, Apr 9 2022 20:52:01) on WebAssembly VM Type "help", "copyright", "credits" or "license" for more information.
That I can avoid it by running this first though:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115301733 | https://github.com/simonw/datasette/issues/1735#issuecomment-1115301733 | https://api.github.com/repos/simonw/datasette/issues/1735 | IC_kwDOBm6k_c5Ceidl | simonw 9599 | 2022-05-02T19:57:19Z | 2022-05-02T19:59:03Z | OWNER | This code breaks if that setting is 0: It's used here: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette setting to disable threading (for Pyodide) 1223263540 | |
1115288284 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115288284 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5CefLc | simonw 9599 | 2022-05-02T19:40:33Z | 2022-05-02T19:40:33Z | OWNER | I'll release this as a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115283922 | https://github.com/simonw/datasette/issues/1734#issuecomment-1115283922 | https://api.github.com/repos/simonw/datasette/issues/1734 | IC_kwDOBm6k_c5CeeHS | simonw 9599 | 2022-05-02T19:35:32Z | 2022-05-02T19:35:32Z | OWNER | I'll use my original from 2009: https://www.djangosnippets.org/snippets/1431/ |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Remove python-baseconv dependency 1223241647 | |
1115282773 | https://github.com/simonw/datasette/issues/1734#issuecomment-1115282773 | https://api.github.com/repos/simonw/datasette/issues/1734 | IC_kwDOBm6k_c5Ced1V | simonw 9599 | 2022-05-02T19:34:15Z | 2022-05-02T19:34:15Z | OWNER | I'm going to vendor it and update the documentation. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Remove python-baseconv dependency 1223241647 | |
1115278325 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115278325 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5Cecv1 | simonw 9599 | 2022-05-02T19:29:05Z | 2022-05-02T19:29:05Z | OWNER | I'm going to add a Datasette setting to disable threading entirely, designed for usage in this particular case. I thought about adding a new setting, then I noticed this:
I'm going to let users set that to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115268245 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115268245 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5CeaSV | simonw 9599 | 2022-05-02T19:18:11Z | 2022-05-02T19:18:11Z | OWNER | Maybe I can leave ```pycon Welcome to the Pyodide terminal emulator 🐍 Python 3.10.2 (main, Apr 9 2022 20:52:01) on WebAssembly VM Type "help", "copyright", "credits" or "license" for more information.
But it looks like i can address this issue just by making |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115262218 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115262218 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5CeY0K | simonw 9599 | 2022-05-02T19:11:51Z | 2022-05-02T19:14:01Z | OWNER | Here's the full diff I applied to Datasette to get it fully working in Pyodide: And as a visible diff: ```diff diff --git a/datasette/app.py b/datasette/app.py index d269372..6c0c5fc 100644 --- a/datasette/app.py +++ b/datasette/app.py @@ -15,7 +15,6 @@ import pkg_resources import re import secrets import sys -import threading import traceback import urllib.parse from concurrent import futures @@ -26,7 +25,6 @@ from itsdangerous import URLSafeSerializer from jinja2 import ChoiceLoader, Environment, FileSystemLoader, PrefixLoader from jinja2.environment import Template from jinja2.exceptions import TemplateNotFound -import uvicorn from .views.base import DatasetteError, ureg from .views.database import DatabaseDownload, DatabaseView @@ -813,7 +811,6 @@ class Datasette: }, "datasette": datasette_version, "asgi": "3.0", - "uvicorn": uvicorn.version, "sqlite": { "version": sqlite_version, "fts_versions": fts_versions, @@ -854,23 +851,7 @@ class Datasette: ]
from .tracer import trace @@ -21,8 +20,6 @@ from .utils import ( ) from .inspect import inspect_hash -connections = threading.local()AttachedDatabase = namedtuple("AttachedDatabase", ("seq", "name", "file")) @@ -43,12 +40,12 @@ class Database: self.hash = None self.cached_size = None self._cached_table_counts = None - self._write_thread = None - self._write_queue = None if not self.is_mutable and not self.is_memory: p = Path(path) self.hash = inspect_hash(p) self.cached_size = p.stat().st_size + self._read_connection = None + self._write_connection = None
@@ -134,60 +131,17 @@ class Database: return results
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115260999 | https://github.com/simonw/datasette/issues/1734#issuecomment-1115260999 | https://api.github.com/repos/simonw/datasette/issues/1734 | IC_kwDOBm6k_c5CeYhH | simonw 9599 | 2022-05-02T19:10:34Z | 2022-05-02T19:10:34Z | OWNER | This is actually mostly a documentation thing: here: https://docs.datasette.io/en/0.61.1/authentication.html#including-an-expiry-time In the code it's only used in these two places: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Remove python-baseconv dependency 1223241647 | |
1115258737 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115258737 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5CeX9x | simonw 9599 | 2022-05-02T19:08:17Z | 2022-05-02T19:08:17Z | OWNER | I was going to vendor I used https://cs.github.com/ and as far as I can tell there aren't any! So I'm going to remove that dependency and work out a smarter way to do this - probably by providing a utility function within Datasette itself. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115256318 | https://github.com/simonw/datasette/issues/1733#issuecomment-1115256318 | https://api.github.com/repos/simonw/datasette/issues/1733 | IC_kwDOBm6k_c5CeXX- | simonw 9599 | 2022-05-02T19:05:55Z | 2022-05-02T19:05:55Z | OWNER | I released a
Datasette never actually sets that cookie itself - it instead encourages plugins to set it in the authentication documentation here: https://docs.datasette.io/en/0.61.1/authentication.html#including-an-expiry-time |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Get Datasette compatible with Pyodide 1223234932 | |
1115196863 | https://github.com/simonw/sqlite-utils/pull/429#issuecomment-1115196863 | https://api.github.com/repos/simonw/sqlite-utils/issues/429 | IC_kwDOCGYnMM5CeI2_ | simonw 9599 | 2022-05-02T18:03:47Z | 2022-05-02T18:52:42Z | OWNER | I made a build of this branch and tested it like this: https://pyodide.org/en/stable/console.html ```pycon
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Depend on click-default-group-wheel 1223177069 | |
1115197644 | https://github.com/simonw/sqlite-utils/pull/429#issuecomment-1115197644 | https://api.github.com/repos/simonw/sqlite-utils/issues/429 | IC_kwDOCGYnMM5CeJDM | simonw 9599 | 2022-05-02T18:04:28Z | 2022-05-02T18:04:28Z | OWNER | I'm going to ship this straight away as |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Depend on click-default-group-wheel 1223177069 | |
1114058210 | https://github.com/simonw/datasette/issues/1727#issuecomment-1114058210 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CZy3i | simonw 9599 | 2022-04-30T21:39:34Z | 2022-04-30T21:39:34Z | OWNER | Something to consider if I look into subprocesses for parallel query execution: https://sqlite.org/howtocorrupt.html#carrying_an_open_database_connection_across_a_fork
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1114038259 | https://github.com/simonw/datasette/issues/1729#issuecomment-1114038259 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CZt_z | simonw 9599 | 2022-04-30T19:06:03Z | 2022-04-30T19:06:03Z | OWNER |
I think |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1114036946 | https://github.com/simonw/datasette/issues/1729#issuecomment-1114036946 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CZtrS | simonw 9599 | 2022-04-30T18:56:25Z | 2022-04-30T19:04:03Z | OWNER | Related: - #1558 Which talks about how there was confusion in this example: https://latest.datasette.io/fixtures/facetable.json?_facet=created&_facet_date=created&_facet=tags&_facet_array=tags&_nosuggest=1&_size=0 Which I fixed in #625 by introducing |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1114037521 | https://github.com/simonw/datasette/issues/1729#issuecomment-1114037521 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CZt0R | simonw 9599 | 2022-04-30T19:01:07Z | 2022-04-30T19:01:07Z | OWNER | I had to look up what That's a bit of a weird thing to expose in the API. Maybe change that to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1114013757 | https://github.com/simonw/datasette/issues/1729#issuecomment-1114013757 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CZoA9 | simonw 9599 | 2022-04-30T16:15:51Z | 2022-04-30T18:54:39Z | OWNER | Deployed a preview of this here: https://latest-1-0-alpha.datasette.io/ Examples:
Second example produces:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112889800 | https://github.com/simonw/datasette/issues/1727#issuecomment-1112889800 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CVVnI | simonw 9599 | 2022-04-29T05:29:38Z | 2022-04-29T05:29:38Z | OWNER | OK, I just got the most incredible result with that! I started up a container running
And... the parallel one beat the non-parallel one decisively, on multiple page refreshes! Not parallel: 77ms Parallel: 47ms So yeah, I'm very confident this is a problem with the GIL. And I am absolutely stunned that @colesbury's fork ran Datasette (which has some reasonably tricky threading and async stuff going on) out of the box! |
{ "total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1112879463 | https://github.com/simonw/datasette/issues/1727#issuecomment-1112879463 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CVTFn | simonw 9599 | 2022-04-29T05:03:58Z | 2022-04-29T05:03:58Z | OWNER | It would be really fun to try running this with the in-development There's a Docker container for it: https://hub.docker.com/r/nogil/python It suggests you can run something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1112878955 | https://github.com/simonw/datasette/issues/1727#issuecomment-1112878955 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CVS9r | simonw 9599 | 2022-04-29T05:02:40Z | 2022-04-29T05:02:40Z | OWNER | Here's a very useful (recent) article about how the GIL works and how to think about it: https://pythonspeed.com/articles/python-gil/ - via https://lobste.rs/s/9hj80j/when_python_can_t_thread_deep_dive_into_gil From that article:
That explains what I'm seeing here. I'm pretty convinced now that the reason I'm not getting a performance boost from parallel queries is that there's more time spent in Python code assembling the results than in SQLite C code executing the query. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1112734577 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112734577 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUvtx | simonw 9599 | 2022-04-28T23:08:42Z | 2022-04-28T23:08:42Z | OWNER | That prototype is a very small amount of code so far: ```diff diff --git a/datasette/renderer.py b/datasette/renderer.py index 4508949..b600e1b 100644 --- a/datasette/renderer.py +++ b/datasette/renderer.py @@ -28,6 +28,10 @@ def convert_specific_columns_to_json(rows, columns, json_cols): def json_renderer(args, data, view_name): """Render a response as JSON""" + from pprint import pprint + + pprint(data) + status_code = 200
@@ -43,6 +47,41 @@ def json_renderer(args, data, view_name): if "rows" in data and not value_as_boolean(args.get("_json_infinity", "0")): data["rows"] = [remove_infinites(row) for row in data["rows"]]
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112732563 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112732563 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUvOT | simonw 9599 | 2022-04-28T23:05:03Z | 2022-04-28T23:05:03Z | OWNER | OK, the prototype of this is looking really good - it's very pleasant to use.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112730416 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112730416 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUusw | simonw 9599 | 2022-04-28T23:01:21Z | 2022-04-28T23:01:21Z | OWNER | I'm not sure what to do about the It's not really relevant to table results, since they are paginated whether or not you ask for them to be. It plays a role in query results, where you might run Adding it to every table result and always setting it to I think I'm going to keep it exclusively in the default representation for the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112721321 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112721321 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUsep | simonw 9599 | 2022-04-28T22:44:05Z | 2022-04-28T22:44:14Z | OWNER | I may be able to implement this mostly in the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112717745 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112717745 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUrmx | simonw 9599 | 2022-04-28T22:38:39Z | 2022-04-28T22:39:05Z | OWNER | (I remain keen on the idea of shipping a plugin that restores the old default API shape to people who have written pre-Datasette-1.0 code against it, but I'll tackle that much later. I really like how jQuery has a culture of doing this.) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112717210 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112717210 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUrea | simonw 9599 | 2022-04-28T22:37:37Z | 2022-04-28T22:37:37Z | OWNER | This means I'll add |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112716611 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112716611 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUrVD | simonw 9599 | 2022-04-28T22:36:24Z | 2022-04-28T22:36:24Z | OWNER | Then I'm going to implement the following
I thought about having I'm tempted to add |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112713581 | https://github.com/simonw/datasette/issues/1729#issuecomment-1112713581 | https://api.github.com/repos/simonw/datasette/issues/1729 | IC_kwDOBm6k_c5CUqlt | simonw 9599 | 2022-04-28T22:31:11Z | 2022-04-28T22:31:11Z | OWNER | I'm going to change the default API response to look like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Implement ?_extra and new API design for TableView 1219385669 | |
1112711115 | https://github.com/simonw/datasette/issues/1715#issuecomment-1112711115 | https://api.github.com/repos/simonw/datasette/issues/1715 | IC_kwDOBm6k_c5CUp_L | simonw 9599 | 2022-04-28T22:26:56Z | 2022-04-28T22:26:56Z | OWNER | I'm not going to use
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Refactor TableView to use asyncinject 1212823665 | |
1112668411 | https://github.com/simonw/datasette/issues/1727#issuecomment-1112668411 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CUfj7 | simonw 9599 | 2022-04-28T21:25:34Z | 2022-04-28T21:25:44Z | OWNER | The two most promising theories at the moment, from here and Twitter and the SQLite forum, are:
A couple of ways to research the in-memory theory:
I need to do some more, better benchmarks using these different approaches. https://twitter.com/laurencerowe/status/1519780174560169987 also suggests:
I like that second idea a lot - I could use the mandelbrot example from https://www.sqlite.org/lang_with.html#outlandish_recursive_query_examples |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111726586 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111726586 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQ5n6 | simonw 9599 | 2022-04-28T04:17:16Z | 2022-04-28T04:19:31Z | OWNER | I could experiment with the Code examples: https://cs.github.com/?scopeName=All+repos&scope=&q=run_in_executor+ProcessPoolExecutor |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111725638 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111725638 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQ5ZG | simonw 9599 | 2022-04-28T04:15:15Z | 2022-04-28T04:15:15Z | OWNER | Useful theory from Keith Medcalf https://sqlite.org/forum/forumpost/e363c69d3441172e
So maybe this is a GIL thing. I should test with some expensive SQL queries (maybe big aggregations against large tables) and see if I can spot an improvement there. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111714665 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111714665 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ2tp | simonw 9599 | 2022-04-28T03:52:47Z | 2022-04-28T03:52:58Z | OWNER | Nice custom template/theme! Yeah, for that I'd recommend hosting elsewhere - on a regular VPS (I use |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111708206 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111708206 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ1Iu | simonw 9599 | 2022-04-28T03:38:56Z | 2022-04-28T03:38:56Z | OWNER | In terms of this bug, there are a few potential fixes:
I'm not keen on that last one because it would be frustrating if you couldn't launch Datasette just because you had an old canned query lying around in your metadata file. So I'm leaning towards option 2. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111707384 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111707384 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ074 | simonw 9599 | 2022-04-28T03:36:46Z | 2022-04-28T03:36:56Z | OWNER | A more realistic solution (which I've been using on several of my own projects) is to keep the data itself in GitHub and encourage users to edit it there - using the GitHub web interface to edit YAML files or similar. Needs your users to be comfortable hand-editing YAML though! You can at least guard against critical errors by having CI run tests against their YAML before deploying. I have a dream of building a more friendly web forms interface which edits the YAML back on GitHub for the user, but that's just a concept at the moment. Even more fun would be if a user-friendly form could submit PRs for review without the user having to know what a PR is! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111706519 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111706519 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ0uX | simonw 9599 | 2022-04-28T03:34:49Z | 2022-04-28T03:34:49Z | OWNER | I've wanted to do stuff like that on Cloud Run too. So far I've assumed that it's not feasible, but recently I've been wondering how hard it would be to have a small (like less than 100KB or so) Datasette instance which persists data to a backing GitHub repository such that when it starts up it can pull the latest copy and any time someone edits it can push their changes. I'm still not sure it would work well on Cloud Run due to the uncertainty at what would happen if Cloud Run decided to boot up a second instance - but it's still an interesting thought exercise. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111705069 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111705069 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ0Xt | simonw 9599 | 2022-04-28T03:31:33Z | 2022-04-28T03:31:33Z | OWNER | Confirmed - this is a bug where immutable databases fail to show a useful error if you write to them with a canned query. Steps to reproduce:
Now do this instead:
And I'm getting a broken error: |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111699175 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111699175 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQy7n | simonw 9599 | 2022-04-28T03:19:48Z | 2022-04-28T03:20:08Z | OWNER | I ran The area on the right is the threads running the DB queries: Interactive version here: https://static.simonwillison.net/static/2022/datasette-parallel-profile.svg |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111698307 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111698307 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQyuD | simonw 9599 | 2022-04-28T03:18:02Z | 2022-04-28T03:18:02Z | OWNER | If the behaviour you are seeing is because the database is running in immutable mode then that's a bug - you should get a useful error message instead! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111697985 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111697985 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQypB | simonw 9599 | 2022-04-28T03:17:20Z | 2022-04-28T03:17:20Z | OWNER | How did you deploy to Cloud Run?
That's why I upgraded |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111683539 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111683539 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQvHT | simonw 9599 | 2022-04-28T02:47:57Z | 2022-04-28T02:47:57Z | OWNER | Maybe this is the Python GIL after all? I've been hoping that the GIL won't be an issue because the So I've been hoping this means that SQLite code itself can run concurrently on multiple cores even when Python threads cannot. But maybe I'm misunderstanding how that works? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111681513 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111681513 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQunp | simonw 9599 | 2022-04-28T02:44:26Z | 2022-04-28T02:44:26Z | OWNER | I could try |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111661331 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111661331 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQpsT | simonw 9599 | 2022-04-28T02:07:31Z | 2022-04-28T02:07:31Z | OWNER | Asked on the SQLite forum about this here: https://sqlite.org/forum/forumpost/ffbfa9f38e |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111602802 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111602802 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQbZy | simonw 9599 | 2022-04-28T00:21:35Z | 2022-04-28T00:21:35Z | OWNER | Tried this but I'm getting back an empty JSON array of traces at the bottom of the page most of the time (intermittently it works correctly): ```diff diff --git a/datasette/database.py b/datasette/database.py index ba594a8..d7f9172 100644 --- a/datasette/database.py +++ b/datasette/database.py @@ -7,7 +7,7 @@ import sys import threading import uuid -from .tracer import trace +from .tracer import trace, trace_child_tasks from .utils import ( detect_fts, detect_primary_keys, @@ -207,30 +207,31 @@ class Database: time_limit_ms = custom_time_limit
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111597176 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111597176 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQaB4 | simonw 9599 | 2022-04-28T00:11:44Z | 2022-04-28T00:11:44Z | OWNER | Though it would be interesting to also have the trace reveal how much time is spent in the functions that wrap that core SQL - the stuff that is being measured at the moment. I have a hunch that this could help solve the over-arching performance mystery. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111595319 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111595319 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQZk3 | simonw 9599 | 2022-04-28T00:09:45Z | 2022-04-28T00:11:01Z | OWNER | Here's where read queries are instrumented: https://github.com/simonw/datasette/blob/7a6654a253dee243518dc542ce4c06dbb0d0801d/datasette/database.py#L241-L242 So the instrumentation is actually capturing quite a bit of Python activity before it gets to SQLite: And then: Ideally I'd like that |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111558204 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111558204 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQQg8 | simonw 9599 | 2022-04-27T22:58:39Z | 2022-04-27T22:58:39Z | OWNER | I should check my timing mechanism. Am I capturing the time taken just in SQLite or does it include time spent in Python crossing between async and threaded world and waiting for a thread pool worker to become available? That could explain the longer query times. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111553029 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111553029 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQPQF | simonw 9599 | 2022-04-27T22:48:21Z | 2022-04-27T22:48:21Z | OWNER | I wonder if it would be worth exploring multiprocessing here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111551076 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111551076 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQOxk | simonw 9599 | 2022-04-27T22:44:51Z | 2022-04-27T22:45:04Z | OWNER | Really wild idea: what if I created three copies of the SQLite database file - as three separate file names - and then balanced the parallel queries across all these? Any chance that could avoid any mysterious locking issues? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111535818 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111535818 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CQLDK | simonw 9599 | 2022-04-27T22:18:45Z | 2022-04-27T22:18:45Z | OWNER | Another avenue: https://twitter.com/weargoggles/status/1519426289920270337
Doesn't look like there's an obvious way to access that from Python via the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111485722 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111485722 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CP-0a | simonw 9599 | 2022-04-27T21:08:20Z | 2022-04-27T21:08:20Z | OWNER | Tried that and it didn't seem to make a difference either. I really need a much deeper view of what's going on here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111462442 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111462442 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CP5Iq | simonw 9599 | 2022-04-27T20:40:59Z | 2022-04-27T20:42:49Z | OWNER | This looks VERY relevant: SQLite Shared-Cache Mode:
Enabled as part of the URI filename:
Turns out I'm already using this for in-memory databases that have |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111460068 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111460068 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CP4jk | simonw 9599 | 2022-04-27T20:38:32Z | 2022-04-27T20:38:32Z | OWNER | WAL mode didn't seem to make a difference. I thought there was a chance it might help multiple read connections operate at the same time but it looks like it really does only matter for when writes are going on. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111456500 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111456500 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CP3r0 | simonw 9599 | 2022-04-27T20:36:01Z | 2022-04-27T20:36:01Z | OWNER | Yeah all of this is pretty much assuming read-only connections. Datasette has a separate mechanism for ensuring that writes are executed one at a time against a dedicated connection from an in-memory queue: - https://github.com/simonw/datasette/issues/682 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111442012 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111442012 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CP0Jc | simonw 9599 | 2022-04-27T20:19:00Z | 2022-04-27T20:19:00Z | OWNER | Something worth digging into: are these parallel queries running against the same SQLite connection or are they each rubbing against a separate SQLite connection? Just realized I know the answer: they're running against separate SQLite connections, because that's how the time limit mechanism works: it installs a progress handler for each connection which terminates it after a set time. This means that if SQLite benefits from multiple threads using the same connection (due to shared caches or similar) then Datasette will not be seeing those benefits. It also means that if there's some mechanism within SQLite that penalizes you for having multiple parallel connections to a single file (just guessing here, maybe there's some kind of locking going on?) then Datasette will suffer those penalties. I should try seeing what happens with WAL mode enabled. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111432375 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111432375 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CPxy3 | simonw 9599 | 2022-04-27T20:07:57Z | 2022-04-27T20:07:57Z | OWNER | Also useful: https://avi.im/blag/2021/fast-sqlite-inserts/ - from a tip on Twitter: https://twitter.com/ricardoanderegg/status/1519402047556235264 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111431785 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111431785 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CPxpp | simonw 9599 | 2022-04-27T20:07:16Z | 2022-04-27T20:07:16Z | OWNER | I think I need some much more in-depth tracing tricks for this. https://www.maartenbreddels.com/perf/jupyter/python/tracing/gil/2021/01/14/Tracing-the-Python-GIL.html looks relevant - uses the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111408273 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111408273 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CPr6R | simonw 9599 | 2022-04-27T19:40:51Z | 2022-04-27T19:42:17Z | OWNER | Relevant: here's the code that sets up a Datasette SQLite connection: https://github.com/simonw/datasette/blob/7a6654a253dee243518dc542ce4c06dbb0d0801d/datasette/database.py#L73-L96 It's using
This is why Datasette reserves a single connection for write queries and queues them up in memory, as described here. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111390433 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111390433 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CPnjh | simonw 9599 | 2022-04-27T19:21:02Z | 2022-04-27T19:21:02Z | OWNER | One weird thing: I noticed that in the parallel trace above the SQL query bars are wider. Mousover shows duration in ms, and I got 13ms for this query:
But in the Given those numbers though I would expect the overall page time to be MUCH worse for the parallel version - but the page load times are instead very close to each other, with parallel often winning. This is super-weird. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111385875 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111385875 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CPmcT | simonw 9599 | 2022-04-27T19:16:57Z | 2022-04-27T19:16:57Z | OWNER | I just remembered the Would explain why the first trace never seems to show more than three SQL queries executing at once. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1111380282 | https://github.com/simonw/datasette/issues/1727#issuecomment-1111380282 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5CPlE6 | simonw 9599 | 2022-04-27T19:10:27Z | 2022-04-27T19:10:27Z | OWNER | Wrote more about that here: https://simonwillison.net/2022/Apr/27/parallel-queries/ Compare https://latest-with-plugins.datasette.io/github/commits?_facet=repo&_facet=committer&_trace=1 With the same thing but with parallel execution disabled: Those total page load time numbers are very similar. Is this parallel optimization worthwhile? Maybe it's only worth it on larger databases? Or maybe larger databases perform worse with this? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1110585475 | https://github.com/simonw/datasette/issues/1724#issuecomment-1110585475 | https://api.github.com/repos/simonw/datasette/issues/1724 | IC_kwDOBm6k_c5CMjCD | simonw 9599 | 2022-04-27T06:15:14Z | 2022-04-27T06:15:14Z | OWNER | Yeah, that page is 438K (but only 20K gzipped). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_trace=1 doesn't work on Global Power Plants demo 1216619276 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
user 1