issue_comments
616 rows where author_association = "CONTRIBUTOR" sorted by node_id
This data as json, CSV (advanced)
Suggested facets: updated_at (date)
issue >30
- Upgrade to CodeMirror 6, add SQL autocomplete 21
- feat: Javascript Plugin API (Custom panels, column menu items with JS actions) 14
- Database page loads too slowly with many large tables (due to table counts) 13
- Stream all results for arbitrary SQL and canned queries 10
- docker image is duplicating db files somehow 10
- Handle spatialite geometry columns better 7
- base_url configuration setting 7
- Exceeding Cloud Run memory limits when deploying a 4.8G database 7
- create-index should run analyze after creating index 7
- Add new spatialite helper methods 7
- Add register_output_renderer hook 6
- Helper methods for working with SpatiaLite 6
- Plugin hook for dynamic metadata 6
- clean checkout & clean environment has test failures 6
- [WIP] Add publish to heroku support 5
- Scripted exports 5
- datasette publish lambda plugin 4
- Build Dockerfile with recent Sqlite + Spatialite 4
- Documentation with recommendations on running Datasette in production without using Docker 4
- bpylist.archiver.CircularReference: archive has a cycle with uid(13) 4
- Add insert --truncate option 4
- Make it easier to insert geometries, with documentation and maybe code 4
- Advanced class-based `conversions=` mechanism 4
- query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data 4
- Proposal: datasette query 4
- Writable canned queries fail with useless non-error against immutable databases 4
- Ability to merge databases and tables 4
- API to insert a single record into an existing table 4
- Exclude virtual tables from datasette inspect 4
- Call for birthday presents: if you're using Datasette, let us know how you're using it here 4
- …
created_at (date) >30 ✖
- 2022-11-16 17
- 2022-10-07 11
- 2019-04-29 9
- 2023-01-25 8
- 2022-02-03 7
- 2018-05-22 6
- 2020-06-23 6
- 2020-07-01 6
- 2022-09-26 6
- 2022-09-27 6
- 2017-11-14 5
- 2019-05-03 5
- 2021-03-23 5
- 2022-01-08 5
- 2023-01-29 5
- 2018-11-05 4
- 2019-06-22 4
- 2020-03-26 4
- 2020-05-10 4
- 2021-01-05 4
- 2021-09-08 4
- 2022-01-13 4
- 2022-03-24 4
- 2022-04-28 4
- 2022-10-01 4
- 2022-11-14 4
- 2022-11-15 4
- 2022-11-17 4
- 2023-06-14 4
- 2023-08-07 4
- …
reactions 7 ✖
- {"total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 589
- {"total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 20
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1} 2
- {"total_count": 1, "+1": 0, "-1": 0, "laugh": 0, "hooray": 1, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 2
- {"total_count": 2, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 1} 1
- {"total_count": 2, "+1": 2, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0} 1
- {"total_count": 3, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 3, "rocket": 0, "eyes": 0} 1
id | html_url | issue_url | node_id ▼ | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
1049879118 | https://github.com/simonw/datasette/issues/1641#issuecomment-1049879118 | https://api.github.com/repos/simonw/datasette/issues/1641 | IC_kwDOBm6k_c4-k-JO | fgregg 536941 | 2022-02-24T13:49:26Z | 2022-02-24T13:49:26Z | CONTRIBUTOR | maybe worth considering adding buttons for paren, asterisk, etc. under the input text box on mobile? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Tweak mobile keyboard settings 1149310456 | |
884910320 | https://github.com/simonw/datasette/issues/1401#issuecomment-884910320 | https://api.github.com/repos/simonw/datasette/issues/1401 | IC_kwDOBm6k_c40vqjw | fgregg 536941 | 2021-07-22T13:26:01Z | 2021-07-22T13:26:01Z | CONTRIBUTOR | ordered lists didn't work either, btw |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
unordered list is not rendering bullet points in description_html on database page 950664971 | |
904982056 | https://github.com/simonw/datasette/issues/859#issuecomment-904982056 | https://api.github.com/repos/simonw/datasette/issues/859 | IC_kwDOBm6k_c418O4o | brandonrobertz 2670795 | 2021-08-24T21:15:04Z | 2021-08-24T21:15:30Z | CONTRIBUTOR | I'm running into issues with this as well. All other pages seem to work with lots of DBs except the home page, which absolutely tanks. Would be willing to put some work into this, if there's been any kind of progress on concepts on how this ought to work. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Database page loads too slowly with many large tables (due to table counts) 642572841 | |
892276385 | https://github.com/simonw/datasette/issues/1419#issuecomment-892276385 | https://api.github.com/repos/simonw/datasette/issues/1419 | IC_kwDOBm6k_c41Lw6h | fgregg 536941 | 2021-08-04T00:58:49Z | 2021-08-04T00:58:49Z | CONTRIBUTOR | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`publish cloudrun` should deploy a more recent SQLite version 959710008 | ||
893114612 | https://github.com/simonw/datasette/issues/1419#issuecomment-893114612 | https://api.github.com/repos/simonw/datasette/issues/1419 | IC_kwDOBm6k_c41O9j0 | fgregg 536941 | 2021-08-05T02:29:06Z | 2021-08-05T02:29:06Z | CONTRIBUTOR | there's a lot of complexity here, that's probably not worth addressing. i got what i needed by patching the dockerfile that cloudrun uses to install a newer version of sqlite. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
`publish cloudrun` should deploy a more recent SQLite version 959710008 | |
895003796 | https://github.com/simonw/datasette/issues/1425#issuecomment-895003796 | https://api.github.com/repos/simonw/datasette/issues/1425 | IC_kwDOBm6k_c41WKyU | abdusco 3243482 | 2021-08-09T07:14:35Z | 2021-08-09T07:14:35Z | CONTRIBUTOR | I believe this also provides a workaround for the problem I face in https://github.com/simonw/datasette/issues/1300. Now I should be able to get table PKs and generate a row URL. I'll test this out and report my findings. ```py from datasette.utils import path_from_row_pks pks = await db.primary_keys(table) url = self.ds.urls.row_blob( database, table, path_from_row_pks(row, pks, not pks), column, ) ``` |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
render_cell() hook should support returning an awaitable 963528457 | |
905899177 | https://github.com/simonw/datasette/issues/859#issuecomment-905899177 | https://api.github.com/repos/simonw/datasette/issues/859 | IC_kwDOBm6k_c41_uyp | brandonrobertz 2670795 | 2021-08-25T21:48:00Z | 2021-08-25T21:48:00Z | CONTRIBUTOR | Upon first stab, there's two issues here: - DB/table/row counts (as discussed above). This isn't too bad if the DBs are actually above the MAX limit check. - Populating the internal DB. On first load of a giant set of DBs, it can take 10-20 mins to populate. By altering datasette and persisting the internal DB to disk, this problem is vastly improved, but I'm sure this will cause problems elsewhere. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Database page loads too slowly with many large tables (due to table counts) 642572841 | |
905904540 | https://github.com/simonw/datasette/issues/859#issuecomment-905904540 | https://api.github.com/repos/simonw/datasette/issues/859 | IC_kwDOBm6k_c41_wGc | brandonrobertz 2670795 | 2021-08-25T21:59:14Z | 2021-08-25T21:59:55Z | CONTRIBUTOR | I did two tests: one with 1000 5-30mb DBs and a second with 20 multi gig DBs. For the second, I created them like so:
This was for deciding whether to use lots of small DBs or to group things into a smaller number of bigger DBs. The second strategy wins. By simply persisting the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Database page loads too slowly with many large tables (due to table counts) 642572841 | |
922363640 | https://github.com/simonw/datasette/issues/1473#issuecomment-922363640 | https://api.github.com/repos/simonw/datasette/issues/1473 | IC_kwDOBm6k_c42-ib4 | mroswell 192568 | 2021-09-18T19:45:47Z | 2021-09-18T19:45:47Z | CONTRIBUTOR | An update, if I remove the Also, if I keep things exactly as they are, and it improperly, but consistently goes to the Weird stuff. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
base logo link visits `undefined` rather than href url 999902754 | |
922394999 | https://github.com/simonw/datasette/issues/1473#issuecomment-922394999 | https://api.github.com/repos/simonw/datasette/issues/1473 | IC_kwDOBm6k_c42-qF3 | mroswell 192568 | 2021-09-19T00:44:39Z | 2021-09-19T00:45:32Z | CONTRIBUTOR | I replaced:
I'd still love to know what caused this (and how to troubleshoot to figure it out), so I'll leave it open for a bit, but I do have a functional logo linking to the Hugo home page, at least locally. I'll likely push tomorrow. (Before trying this, I tried to apply a background image to the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
base logo link visits `undefined` rather than href url 999902754 | |
913001282 | https://github.com/simonw/datasette/pull/1455#issuecomment-913001282 | https://api.github.com/repos/simonw/datasette/issues/1455 | IC_kwDOBm6k_c42a0tC | ctb 51016 | 2021-09-04T16:31:24Z | 2021-09-04T16:31:24Z | CONTRIBUTOR | I love it! maybe 'researchers' instead? Or 'scientists and researchers'? |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add scientists to target groups 988325628 | |
915279711 | https://github.com/simonw/datasette/issues/1464#issuecomment-915279711 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42jg9f | ctb 51016 | 2021-09-08T14:16:49Z | 2021-09-08T14:16:49Z | CONTRIBUTOR | on commit d57ab156b35ec642 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
915299013 | https://github.com/simonw/datasette/issues/1464#issuecomment-915299013 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42jlrF | bobwhitelock 7476523 | 2021-09-08T14:40:28Z | 2021-09-08T14:40:28Z | CONTRIBUTOR | What are the full errors you're getting? This may be the same issue as described in https://github.com/simonw/datasette/pull/1223 - essentially the test suite (and corresponding Datasette features I assume) are by default implicitly dependent on your Sqlite installation having been compiled with the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
915302885 | https://github.com/simonw/datasette/issues/1464#issuecomment-915302885 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42jmnl | ctb 51016 | 2021-09-08T14:44:50Z | 2021-09-08T14:44:50Z | CONTRIBUTOR | thanks for the response! full errors attached; excerpt: ``` ...
/Users/t/dev/datasette/tests/test_api.py:1115: AssertionError ``` A quick scan of #1223 suggests you're right. Unfortunately, pysqlite3-binary isn't available for Mac OS X, so I can't quickly check that that fixes it; will do so later. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
915343886 | https://github.com/simonw/datasette/issues/1464#issuecomment-915343886 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42jwoO | bobwhitelock 7476523 | 2021-09-08T15:32:06Z | 2021-09-08T15:32:06Z | CONTRIBUTOR | Thanks, that does look similar!
Ah that makes sense, I guess that's why this isn't just always installed already. I wonder if a possible solution to this issue could be doing feature detection on whether this feature is supported by the current Sqlite version, and if not these tests could be disabled locally? But possibly there's a better way to handle this, will see what @simonw thinks |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
917642487 | https://github.com/simonw/datasette/issues/1464#issuecomment-917642487 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42shz3 | ctb 51016 | 2021-09-12T14:03:09Z | 2021-09-12T14:03:09Z | CONTRIBUTOR | haven't had time to get back to this, but idle thought that I'm recording for later investigation: how does the continuous integration handle this installation issue? Is it documented there? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
918621705 | https://github.com/simonw/datasette/issues/1464#issuecomment-918621705 | https://api.github.com/repos/simonw/datasette/issues/1464 | IC_kwDOBm6k_c42wQ4J | bobwhitelock 7476523 | 2021-09-13T22:17:17Z | 2021-09-13T22:17:17Z | CONTRIBUTOR |
Not certain, but I think tests in CI run on Ubuntu and don't appear to install any additional Sqlite-related dependencies, and so my guess is the version of Sqlite installed by default on Ubuntu has the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
clean checkout & clean environment has test failures 991191951 | |
919135732 | https://github.com/simonw/datasette/pull/1453#issuecomment-919135732 | https://api.github.com/repos/simonw/datasette/issues/1453 | IC_kwDOBm6k_c42yOX0 | dependabot[bot] 49699333 | 2021-09-14T13:10:38Z | 2021-09-14T13:10:38Z | CONTRIBUTOR | Superseded by #1471. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump black from 21.7b0 to 21.8b0 982780906 | |
938171377 | https://github.com/simonw/datasette/issues/1480#issuecomment-938171377 | https://api.github.com/repos/simonw/datasette/issues/1480 | IC_kwDOBm6k_c4361vx | ghing 110420 | 2021-10-07T21:33:12Z | 2021-10-07T21:33:12Z | CONTRIBUTOR | Thanks for the reply @simonw. What services have you had better success with than Cloud Run for larger database? Also, what about my issue description makes you think there may be a workaround? Is there any instrumentation I could add to see at which point in the deploy the memory usage spikes? Should I be able to see this whether it's running under Docker locally, or do you suspect this is Cloud Run-specific? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exceeding Cloud Run memory limits when deploying a 4.8G database 1015646369 | |
953334718 | https://github.com/simonw/datasette/issues/1380#issuecomment-953334718 | https://api.github.com/repos/simonw/datasette/issues/1380 | IC_kwDOBm6k_c440ru- | glasnt 813732 | 2021-10-27T21:45:04Z | 2021-10-27T21:45:04Z | CONTRIBUTOR | I am also getting this issue, using the currently most recent version of datasette
If I run
Adding new files while datasette is running shows no new files, and removing files causes datasette to return 500 errors.
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Serve all db files in a folder 924748955 | |
953366110 | https://github.com/simonw/datasette/issues/1380#issuecomment-953366110 | https://api.github.com/repos/simonw/datasette/issues/1380 | IC_kwDOBm6k_c440zZe | glasnt 813732 | 2021-10-27T22:48:55Z | 2021-10-27T22:48:55Z | CONTRIBUTOR | It looks like if the files argument is a directory, I tried seeing if I could get
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Serve all db files in a folder 924748955 | |
954384496 | https://github.com/simonw/datasette/pull/1495#issuecomment-954384496 | https://api.github.com/repos/simonw/datasette/issues/1495 | IC_kwDOBm6k_c444sBw | fgregg 536941 | 2021-10-29T03:07:13Z | 2021-10-29T03:07:13Z | CONTRIBUTOR | okay @simonw, made the requested changes. tests are running locally. i think this is ready for you to look at again. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow routes to have extra options 1033678984 | |
943594735 | https://github.com/simonw/datasette/pull/1489#issuecomment-943594735 | https://api.github.com/repos/simonw/datasette/issues/1489 | IC_kwDOBm6k_c44Phzv | dependabot[bot] 49699333 | 2021-10-14T18:04:12Z | 2021-10-14T18:04:12Z | CONTRIBUTOR | Looks like this PR is closed. If you re-open it I'll rebase it as long as no-one else has edited it (you can use |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update pyyaml requirement from ~=5.3 to >=5.3,<7.0 1026379132 | |
943594738 | https://github.com/simonw/datasette/pull/1489#issuecomment-943594738 | https://api.github.com/repos/simonw/datasette/issues/1489 | IC_kwDOBm6k_c44Phzy | dependabot[bot] 49699333 | 2021-10-14T18:04:13Z | 2021-10-14T18:04:13Z | CONTRIBUTOR | OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting If you change your mind, just re-open this PR and I'll resolve any conflicts on it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update pyyaml requirement from ~=5.3 to >=5.3,<7.0 1026379132 | |
946255239 | https://github.com/simonw/datasette/issues/1432#issuecomment-946255239 | https://api.github.com/repos/simonw/datasette/issues/1432 | IC_kwDOBm6k_c44ZrWH | mroswell 192568 | 2021-10-18T23:55:25Z | 2021-10-18T23:55:25Z | CONTRIBUTOR | I am getting this when I visit my live Datasette page:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rename Datasette.__init__(config=) parameter to settings= 969855774 | |
946287922 | https://github.com/simonw/datasette/issues/1432#issuecomment-946287922 | https://api.github.com/repos/simonw/datasette/issues/1432 | IC_kwDOBm6k_c44ZzUy | mroswell 192568 | 2021-10-19T01:16:41Z | 2021-10-19T01:16:41Z | CONTRIBUTOR | Resolved, with assistance from @ashishdotme (Thank you!) Updated requirements.txt to include:
Ran:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Rename Datasette.__init__(config=) parameter to settings= 969855774 | |
946467547 | https://github.com/simonw/datasette/issues/1396#issuecomment-946467547 | https://api.github.com/repos/simonw/datasette/issues/1396 | IC_kwDOBm6k_c44afLb | MichaelTiemannOSC 72577720 | 2021-10-19T08:10:26Z | 2021-10-19T08:10:26Z | CONTRIBUTOR | Now that 0.59 has excellent annotated release notes, you can re-confirm this is fixed by updating the published Docker image and checking that these fixes still work ;-) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
"invalid reference format" publishing Docker image 944903881 | |
947196177 | https://github.com/simonw/datasette/issues/1480#issuecomment-947196177 | https://api.github.com/repos/simonw/datasette/issues/1480 | IC_kwDOBm6k_c44dRER | ghing 110420 | 2021-10-20T00:05:10Z | 2021-10-20T00:05:10Z | CONTRIBUTOR | I was looking through the Dockerfile-generation code to see if there was anything that would cause memory usage to be a lot during deployment. I noticed that the Dockerfile runs Or would that come into play when running |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exceeding Cloud Run memory limits when deploying a 4.8G database 1015646369 | |
947203725 | https://github.com/simonw/datasette/issues/1480#issuecomment-947203725 | https://api.github.com/repos/simonw/datasette/issues/1480 | IC_kwDOBm6k_c44dS6N | ghing 110420 | 2021-10-20T00:21:54Z | 2021-10-20T00:21:54Z | CONTRIBUTOR | This StackOverflow post, sqlite - Cloud Run: Why does my instance need so much RAM?, points to this section of the Cloud Run docs that says:
Does datasette write any large files when starting? Or does the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Exceeding Cloud Run memory limits when deploying a 4.8G database 1015646369 | |
949604763 | https://github.com/simonw/datasette/issues/1284#issuecomment-949604763 | https://api.github.com/repos/simonw/datasette/issues/1284 | IC_kwDOBm6k_c44mdGb | fgregg 536941 | 2021-10-22T12:54:34Z | 2021-10-22T12:54:34Z | CONTRIBUTOR | i'm going to take a swing at this today. we'll see. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Feature or Documentation Request: Individual table as home page template 845794436 | |
950150483 | https://github.com/simonw/datasette/issues/1401#issuecomment-950150483 | https://api.github.com/repos/simonw/datasette/issues/1401 | IC_kwDOBm6k_c44oiVT | jaywgraves 418191 | 2021-10-23T13:09:10Z | 2021-10-23T13:09:10Z | CONTRIBUTOR | I think it's because of this in
https://github.com/simonw/datasette/blame/main/datasette/static/app.css#L35-L38 You could probably reinstate that by providing your own CSS. https://docs.datasette.io/en/0.24/custom_templates.html#custom-css-and-javascript |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
unordered list is not rendering bullet points in description_html on database page 950664971 | |
970266123 | https://github.com/simonw/datasette/issues/1012#issuecomment-970266123 | https://api.github.com/repos/simonw/datasette/issues/1012 | IC_kwDOBm6k_c451RYL | bollwyvl 45380 | 2021-11-16T13:18:36Z | 2021-11-16T13:18:36Z | CONTRIBUTOR | Congratulations, looks like it went through! There was a bit of a hold-up on the JupyterLab ones, but it's semi automated: a dependabot pr to warehouse and a CI deploy, with a click in between. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
For 1.0 update trove classifier in setup.py 718540751 | |
971568829 | https://github.com/simonw/datasette/pull/1500#issuecomment-971568829 | https://api.github.com/repos/simonw/datasette/issues/1500 | IC_kwDOBm6k_c456Pa9 | dependabot[bot] 49699333 | 2021-11-17T13:13:58Z | 2021-11-17T13:13:58Z | CONTRIBUTOR | Superseded by #1514. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump black from 21.9b0 to 21.10b0 1041158024 | |
960420237 | https://github.com/simonw/datasette/pull/1495#issuecomment-960420237 | https://api.github.com/repos/simonw/datasette/issues/1495 | IC_kwDOBm6k_c45PtmN | mroswell 192568 | 2021-11-04T03:12:01Z | 2021-11-04T03:12:01Z | CONTRIBUTOR | This all looks promising! I will need detailed documentation on how to upgrade datasette once it's available, and how to implement. (@fgregg example looks very straightforward on the plugin front.) I'll be so excited if I can get: https://list.saferdisinfectants.org/ instead of https://list.saferdisinfectants.org/disinfectants/listN |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow routes to have extra options 1033678984 | |
972852184 | https://github.com/simonw/datasette/pull/1514#issuecomment-972852184 | https://api.github.com/repos/simonw/datasette/issues/1514 | IC_kwDOBm6k_c45_IvY | dependabot[bot] 49699333 | 2021-11-18T13:11:15Z | 2021-11-18T13:11:15Z | CONTRIBUTOR | Superseded by #1516. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump black from 21.9b0 to 21.11b0 1056117435 | |
967747190 | https://github.com/simonw/datasette/issues/1380#issuecomment-967747190 | https://api.github.com/repos/simonw/datasette/issues/1380 | IC_kwDOBm6k_c45rqZ2 | glasnt 813732 | 2021-11-13T00:47:26Z | 2021-11-13T00:47:26Z | CONTRIBUTOR | Would it make sense to run datasette with a fswatch/inotifywait on a folder, then? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Serve all db files in a folder 924748955 | |
974108455 | https://github.com/simonw/datasette/pull/1495#issuecomment-974108455 | https://api.github.com/repos/simonw/datasette/issues/1495 | IC_kwDOBm6k_c46D7cn | mroswell 192568 | 2021-11-19T14:14:35Z | 2021-11-19T14:14:35Z | CONTRIBUTOR | A nudge on this. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow routes to have extra options 1033678984 | |
975955589 | https://github.com/simonw/datasette/issues/1528#issuecomment-975955589 | https://api.github.com/repos/simonw/datasette/issues/1528 | IC_kwDOBm6k_c46K-aF | asg017 15178711 | 2021-11-22T22:00:30Z | 2021-11-22T22:00:30Z | CONTRIBUTOR | Oh, another thing to consider: I believe this would be the first |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new `"sql_file"` key to Canned Queries in metadata? 1060631257 | |
976117989 | https://github.com/simonw/datasette/issues/1522#issuecomment-976117989 | https://api.github.com/repos/simonw/datasette/issues/1522 | IC_kwDOBm6k_c46LmDl | glasnt 813732 | 2021-11-23T03:00:34Z | 2021-11-23T03:00:34Z | CONTRIBUTOR | I tried deploying the most recent version of the Dockerfile in this thread (link to comment), and after trying a few different different combinations, I was only successful when I used Using this method, I got a very similar issue to you: The first time I'd load the site I'd get a 503. But after that first load, I didn't get the issue again. It would re-occur if the service started from cold boot. I suspect this is a race condition in the supervisord configuration. The errors I got were the same Looking at the order of logs getting back, the processes reported successfully completing loading after the first 503 was returned, so that makes me think race condition. I can replicate this locally, if I Unfortunately supervisor/supervisor issue 122 (not linking as to prevent cross-project link spam) seems to say that dependency chaining is a feature that's been asked for for a long time, but hasn't been implemented. You could try some suggestions in that thread. |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Deploy a live instance of demos/apache-proxy 1058896236 | |
991754237 | https://github.com/simonw/datasette/issues/1549#issuecomment-991754237 | https://api.github.com/repos/simonw/datasette/issues/1549 | IC_kwDOBm6k_c47HPf9 | fgregg 536941 | 2021-12-11T19:14:39Z | 2021-12-11T19:14:39Z | CONTRIBUTOR | that option is not available on custom queries. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Redesign CSV export to improve usability 1077620955 | |
992971072 | https://github.com/simonw/datasette/issues/526#issuecomment-992971072 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c47L4lA | fgregg 536941 | 2021-12-13T22:29:34Z | 2021-12-13T22:29:34Z | CONTRIBUTOR | just came by to open this issue. would make my data analysis in observable a lot better! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
992986587 | https://github.com/simonw/datasette/issues/1553#issuecomment-992986587 | https://api.github.com/repos/simonw/datasette/issues/1553 | IC_kwDOBm6k_c47L8Xb | fgregg 536941 | 2021-12-13T22:57:04Z | 2021-12-13T22:57:04Z | CONTRIBUTOR | would also be good if the header said the what the max row limit was |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
if csv export is truncated in non streaming mode set informative response header 1079111498 | |
993014772 | https://github.com/simonw/datasette/issues/1553#issuecomment-993014772 | https://api.github.com/repos/simonw/datasette/issues/1553 | IC_kwDOBm6k_c47MDP0 | fgregg 536941 | 2021-12-13T23:46:18Z | 2021-12-13T23:46:18Z | CONTRIBUTOR | these headers would also be relevant for json exports of custom queries |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
if csv export is truncated in non streaming mode set informative response header 1079111498 | |
993078038 | https://github.com/simonw/datasette/issues/526#issuecomment-993078038 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c47MSsW | fgregg 536941 | 2021-12-14T01:46:52Z | 2021-12-14T01:46:52Z | CONTRIBUTOR | the nested query idea is very nice, and i stole if for my client side paginator. However, it won't do the right thing if the original query orders by random(). If you go the nested query route, maybe raise a 4XX status code if the query has such a clause? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
995296725 | https://github.com/simonw/datasette/issues/1552#issuecomment-995296725 | https://api.github.com/repos/simonw/datasette/issues/1552 | IC_kwDOBm6k_c47UwXV | davidbgk 3556 | 2021-12-15T23:29:32Z | 2021-12-15T23:29:32Z | CONTRIBUTOR | @simonw thank you for your fast answer and your guidance! While digging into the code, I found an undocumented way of doing it:
The only remaining problem with that solution is here: https://github.com/simonw/datasette/blob/250db8192cb8aba5eb8cd301ccc2a49525bc3d24/datasette/facets.py#L33 We have:
But it requires to cast the
I guess it's an unspotted bug? (I mean, independently of the facets-with-arrays issue.) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow to set `facets_array` in metadata (like current `facets`) 1078702875 | |
996229007 | https://github.com/simonw/datasette/issues/1552#issuecomment-996229007 | https://api.github.com/repos/simonw/datasette/issues/1552 | IC_kwDOBm6k_c47YT-P | davidbgk 3556 | 2021-12-16T22:04:39Z | 2021-12-16T22:04:39Z | CONTRIBUTOR | Wow, that was fast, thank you so much @simonw !
I agree that it's not ideal, my initial naive approach was to detect if it's an array, like what is done here: But it requires an extra query to determine the type, which is a bit problematic, especially for big tables I guess. Taking a look at #510, I wonder if a To be consistent with the Throwing a lot of ideas without knowing the big picture… but sometimes newcomers have superpowers :). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Allow to set `facets_array` in metadata (like current `facets`) 1078702875 | |
997128712 | https://github.com/simonw/datasette/issues/1561#issuecomment-997128712 | https://api.github.com/repos/simonw/datasette/issues/1561 | IC_kwDOBm6k_c47bvoI | fgregg 536941 | 2021-12-18T02:35:48Z | 2021-12-18T02:35:48Z | CONTRIBUTOR | interesting! i love this feature. this + full caching with cloudflare is really super! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
add hash id to "_memory" url if hashed url mode is turned on and crossdb is also turned on 1082765654 | |
997511968 | https://github.com/simonw/datasette/issues/1547#issuecomment-997511968 | https://api.github.com/repos/simonw/datasette/issues/1547 | IC_kwDOBm6k_c47dNMg | wragge 127565 | 2021-12-20T01:21:59Z | 2021-12-20T01:21:59Z | CONTRIBUTOR | I've installed the alpha version but get an error when starting up Datasette:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail to load custom templates 1076388044 | |
997519202 | https://github.com/simonw/datasette/issues/1547#issuecomment-997519202 | https://api.github.com/repos/simonw/datasette/issues/1547 | IC_kwDOBm6k_c47dO9i | wragge 127565 | 2021-12-20T01:36:58Z | 2021-12-20T01:36:58Z | CONTRIBUTOR | Yep, that works -- thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail to load custom templates 1076388044 | |
1002825217 | https://github.com/simonw/datasette/issues/1583#issuecomment-1002825217 | https://api.github.com/repos/simonw/datasette/issues/1583 | IC_kwDOBm6k_c47xeYB | fgregg 536941 | 2021-12-30T00:34:16Z | 2021-12-30T00:34:16Z | CONTRIBUTOR | if that is not desirable, it might be good to document that users might want to set up a lifecycle rule to automatically delete these build artifacts. something like https://stackoverflow.com/questions/59937542/can-i-delete-container-images-from-google-cloud-storage-artifacts-bucket |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
consider adding deletion step of cloudbuild artifacts to gcloud publish 1090810196 | |
1021497165 | https://github.com/simonw/datasette/issues/1612#issuecomment-1021497165 | https://api.github.com/repos/simonw/datasette/issues/1612 | IC_kwDOBm6k_c484s9N | jsfenfen 639012 | 2022-01-25T18:44:23Z | 2022-01-25T18:44:23Z | CONTRIBUTOR | OMG, this might be the fastest OS ticket I've ever filed, thanks so much @simonw |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Move canned queries closer to the SQL input area 1114147905 | |
1007844190 | https://github.com/simonw/datasette/pull/1574#issuecomment-1007844190 | https://api.github.com/repos/simonw/datasette/issues/1574 | IC_kwDOBm6k_c48Ente | fgregg 536941 | 2022-01-08T00:42:12Z | 2022-01-08T00:42:12Z | CONTRIBUTOR | is there a reason to not always use the slim option? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
introduce new option for datasette package to use a slim base image 1084193403 | |
1010947634 | https://github.com/simonw/datasette/issues/1591#issuecomment-1010947634 | https://api.github.com/repos/simonw/datasette/issues/1591 | IC_kwDOBm6k_c48QdYy | psychemedia 82988 | 2022-01-12T11:32:17Z | 2022-01-12T11:32:17Z | CONTRIBUTOR | Is it possible to parse things like |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Maybe let plugins define custom serve options? 1100015398 | |
1016651485 | https://github.com/simonw/datasette/issues/1601#issuecomment-1016651485 | https://api.github.com/repos/simonw/datasette/issues/1601 | IC_kwDOBm6k_c48mN7d | eyeseast 25778 | 2022-01-19T16:39:03Z | 2022-01-19T16:39:03Z | CONTRIBUTOR | I think both of these are Spatialite specific. They get generated when you first initialize the extension. KNN is actually deprecated in favor of KNN2, as I understand it. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add KNN and data_licenses to hidden tables list 1105916061 | |
1016994329 | https://github.com/simonw/datasette/issues/1605#issuecomment-1016994329 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48nhoZ | eyeseast 25778 | 2022-01-20T00:27:17Z | 2022-01-20T00:27:17Z | CONTRIBUTOR | Right now, I usually have a line in a Makefile like this:
That all assumes I've loaded whatever I need into It works, but as you can see, it's a lot to manage, a lot of boilerplate, and it wasn't obvious how to get there. If there's an error in the canned query, I get an HTML error page, so that's hard to debug. And it's only one query, so each output needs a line like this. Make isn't ideal, either, for that reason. The thing I really liked with
And then do:
For HTML export, maybe there's a |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1018741262 | https://github.com/simonw/datasette/issues/1605#issuecomment-1018741262 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48uMIO | eyeseast 25778 | 2022-01-21T18:05:09Z | 2022-01-21T18:05:09Z | CONTRIBUTOR | Thinking about this more, as well as #1356 and various other tickets related to output formats, I think there's a missing plugin hook for formatting results, separate from Right now, as I understand it, getting output in any format goes through the normal view stack -- a table, a row or a query -- and so by the time It's possible this could be done using Another option is some kind of registry of serializers, which |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1018778667 | https://github.com/simonw/datasette/issues/1605#issuecomment-1018778667 | https://api.github.com/repos/simonw/datasette/issues/1605 | IC_kwDOBm6k_c48uVQr | eyeseast 25778 | 2022-01-21T19:00:01Z | 2022-01-21T19:00:01Z | CONTRIBUTOR | Let me know if you want help prototyping any of this, because I'm thinking about it and trying stuff out. Happy to be a sounding board, if it helps. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Scripted exports 1108671952 | |
1031455498 | https://github.com/simonw/datasette/pull/1593#issuecomment-1031455498 | https://api.github.com/repos/simonw/datasette/issues/1593 | IC_kwDOBm6k_c49esMK | dependabot[bot] 49699333 | 2022-02-07T13:13:22Z | 2022-02-07T13:13:22Z | CONTRIBUTOR | Superseded by #1631. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update pytest-asyncio requirement from <0.17,>=0.10 to >=0.10,<0.18 1101705012 | |
1062450649 | https://github.com/simonw/datasette/issues/1655#issuecomment-1062450649 | https://api.github.com/repos/simonw/datasette/issues/1655 | IC_kwDOBm6k_c4_U7XZ | fgregg 536941 | 2022-03-09T01:10:46Z | 2022-03-09T01:10:46Z | CONTRIBUTOR | i increased the max_returned_row, because I have some scripts that get CSVs from this site, and this makes doing pagination of CSVs less annoying for many cases. i know that's streaming csvs is something you are hoping to address in 1.0. let me know if there's anything i can do to help with that. as for what if anything can be done about the size of the dom, I don't have any ideas right now, but i'll poke around. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data 1163369515 | |
1065940779 | https://github.com/simonw/datasette/issues/1384#issuecomment-1065940779 | https://api.github.com/repos/simonw/datasette/issues/1384 | IC_kwDOBm6k_c4_iPcr | brandonrobertz 2670795 | 2022-03-12T18:49:29Z | 2022-03-12T18:50:07Z | CONTRIBUTOR | Hello! Just wanted to chime in and note that there's a plugin to have Datasette watch for updates to an external metadata.yaml/json and update the internal settings accordingly, so I think the cache/poll use case is already covered. @khusmann If you don't need truly dynamic metadata then what you've come up with or the plugin ought to work fine. Making the get_metadata async won't improve the situation by itself as only some of the code paths accessing metadata use that hook. The other paths use the internal metadata dict. Trying to force all paths through a async hook would have performance ramifications and making everything use the internal meta will cause problems for users that need changes to take effect immediately. This is why I came to the non-async solution as it was the path of least change within Datasette. As always, open to new ideas, etc! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for dynamic metadata 930807135 | |
1066006292 | https://github.com/simonw/datasette/issues/1384#issuecomment-1066006292 | https://api.github.com/repos/simonw/datasette/issues/1384 | IC_kwDOBm6k_c4_ifcU | brandonrobertz 2670795 | 2022-03-13T02:09:44Z | 2022-03-13T02:09:44Z | CONTRIBUTOR |
Reading from sqlite DBs is pretty quick and I didn't notice significant performance issues when I was benchmarking. I tested on very large Datasette deployments (hundreds of DBs, millions of rows). See "Many small queries are efficient in sqlite" for more information on the rationale here. Also note that in the datasette-live-config reference plugin, the DB connection is cached, so that eliminated most of the performance worries we had. If you need to ensure fresh metadata is being read inside of a
Yes correct, the datadette-remote-metadata plugin doesn't do that. But the datasette-live-config plugin does. It supports a Good luck! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for dynamic metadata 930807135 | |
1066169718 | https://github.com/simonw/datasette/issues/1384#issuecomment-1066169718 | https://api.github.com/repos/simonw/datasette/issues/1384 | IC_kwDOBm6k_c4_jHV2 | brandonrobertz 2670795 | 2022-03-13T19:48:49Z | 2022-03-13T19:48:49Z | CONTRIBUTOR |
You shouldn't need to do this, as I mentioned previously. The code inside |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for dynamic metadata 930807135 | |
1066222323 | https://github.com/simonw/datasette/issues/1384#issuecomment-1066222323 | https://api.github.com/repos/simonw/datasette/issues/1384 | IC_kwDOBm6k_c4_jULz | brandonrobertz 2670795 | 2022-03-14T00:36:42Z | 2022-03-14T00:36:42Z | CONTRIBUTOR |
All good. Report back any issues you find with this stuff. Metadata/dynamic config hasn't been tested widely outside of what I've done AFAIK. If you find a strong use case for async meta, it's going to be better to know sooner rather than later! |
{ "total_count": 1, "+1": 1, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Plugin hook for dynamic metadata 930807135 | |
1087428593 | https://github.com/simonw/datasette/issues/1549#issuecomment-1087428593 | https://api.github.com/repos/simonw/datasette/issues/1549 | IC_kwDOBm6k_c5A0Nfx | fgregg 536941 | 2022-04-04T11:17:13Z | 2022-04-04T11:17:13Z | CONTRIBUTOR | another way to get the behavior of downloading the file is to use the download attribute of the anchor tag https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a#attr-download |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Redesign CSV export to improve usability 1077620955 | |
1077047152 | https://github.com/simonw/datasette/pull/1582#issuecomment-1077047152 | https://api.github.com/repos/simonw/datasette/issues/1582 | IC_kwDOBm6k_c5AMm9w | fgregg 536941 | 2022-03-24T04:07:58Z | 2022-03-24T04:07:58Z | CONTRIBUTOR | this has been obviated by the datasette-hashed-urls plugin |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
don't set far expiry if hash is '000' 1090055810 | |
1077047295 | https://github.com/simonw/datasette/issues/1581#issuecomment-1077047295 | https://api.github.com/repos/simonw/datasette/issues/1581 | IC_kwDOBm6k_c5AMm__ | fgregg 536941 | 2022-03-24T04:08:18Z | 2022-03-24T04:08:18Z | CONTRIBUTOR | this has been addressed by the datasette-hashed-urls plugin |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
when hashed urls are turned on, the _memory db has improperly long-lived cache expiry 1089529555 | |
1078126065 | https://github.com/simonw/datasette/issues/1684#issuecomment-1078126065 | https://api.github.com/repos/simonw/datasette/issues/1684 | IC_kwDOBm6k_c5AQuXx | fgregg 536941 | 2022-03-24T20:08:56Z | 2022-03-24T20:13:19Z | CONTRIBUTOR | would be nice if the behavior was
This is based on my assumption that what determines autofaceting is the rarity of unique values. Which may not be true! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Mechanism for disabling faceting on large tables only 1179998071 | |
1079550754 | https://github.com/simonw/datasette/issues/1688#issuecomment-1079550754 | https://api.github.com/repos/simonw/datasette/issues/1688 | IC_kwDOBm6k_c5AWKMi | hydrosquall 9020979 | 2022-03-26T01:27:27Z | 2022-03-26T03:16:29Z | CONTRIBUTOR |
As a workaround, I found I can serve my statics from a non-plugin specific folder using the --static CLI flag.
It's not ideal because it means I'll change the cache pattern path depending on how the plugin is running (via pip install or as a one off script), but it's usable as a workaround. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins? 1181432624 | |
1079806857 | https://github.com/simonw/datasette/issues/1688#issuecomment-1079806857 | https://api.github.com/repos/simonw/datasette/issues/1688 | IC_kwDOBm6k_c5AXIuJ | hydrosquall 9020979 | 2022-03-27T01:01:14Z | 2022-03-27T01:01:14Z | CONTRIBUTOR | Thank you! I went through the cookiecutter template, and published my first package here: https://github.com/hydrosquall/datasette-nteract-data-explorer |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
[plugins][documentation] Is it possible to serve per-plugin static folders when writing one-off (single file) plugins? 1181432624 | |
1105588651 | https://github.com/simonw/datasette/issues/1101#issuecomment-1105588651 | https://api.github.com/repos/simonw/datasette/issues/1101 | IC_kwDOBm6k_c5B5fGr | eyeseast 25778 | 2022-04-21T18:15:39Z | 2022-04-21T18:15:39Z | CONTRIBUTOR | What if you split rendering and streaming into two things:
That way current plugins still work, and streaming is purely additive. A |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
1105642187 | https://github.com/simonw/datasette/issues/1101#issuecomment-1105642187 | https://api.github.com/repos/simonw/datasette/issues/1101 | IC_kwDOBm6k_c5B5sLL | eyeseast 25778 | 2022-04-21T18:59:08Z | 2022-04-21T18:59:08Z | CONTRIBUTOR | Ha! That was your idea (and a good one). But it's probably worth measuring to see what overhead it adds. It did require both passing in the database and making the whole thing Just timing the queries themselves:
Looking at the network panel:
I'm not sure how best to time the GeoJSON generation, but it would be interesting to check. Maybe I'll write a plugin to add query times to response headers. The other thing to consider with async streaming is that it might be well-suited for a slower response. When I have to get the whole result and send a response in a fixed amount of time, I need the most efficient query possible. If I can hang onto a connection and get things one chunk at a time, maybe it's ok if there's some overhead. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
register_output_renderer() should support streaming data 749283032 | |
1092357672 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092357672 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHA4o | eyeseast 25778 | 2022-04-08T01:39:40Z | 2022-04-08T01:39:40Z | CONTRIBUTOR |
That's my thinking, too. It's really the thing I've been wanting since writing
I think this probably needs either a new plugin hook separate from |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092370880 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092370880 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHEHA | eyeseast 25778 | 2022-04-08T02:07:40Z | 2022-04-08T02:07:40Z | CONTRIBUTOR | So maybe
And stream gets an iterator, instead of a list of rows, so it can efficiently handle large queries. Maybe it also gets passed a destination stream, or it returns an iterator. I'm not sure what makes more sense. Either way, that might cover both CLI exports and streaming responses. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1092386254 | https://github.com/simonw/datasette/issues/1699#issuecomment-1092386254 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BHH3O | eyeseast 25778 | 2022-04-08T02:39:25Z | 2022-04-08T02:39:25Z | CONTRIBUTOR | And just to think this through a little more, here's what
Alternately, that could be an async generator, like this:
Not sure which makes more sense, but I think this pattern would open up a lot of possibility. If you had your stream_indented_json function, you could do |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1094453751 | https://github.com/simonw/datasette/issues/1699#issuecomment-1094453751 | https://api.github.com/repos/simonw/datasette/issues/1699 | IC_kwDOBm6k_c5BPAn3 | eyeseast 25778 | 2022-04-11T01:32:12Z | 2022-04-11T01:32:12Z | CONTRIBUTOR | Was looking through old issues and realized a bunch of this got discussed in #1101 (including by me!), so sorry to rehash all this. Happy to help with whatever piece of it I can. Would be very excited to be able to use format plugins with exports. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Proposal: datasette query 1193090967 | |
1099540225 | https://github.com/simonw/datasette/issues/1713#issuecomment-1099540225 | https://api.github.com/repos/simonw/datasette/issues/1713 | IC_kwDOBm6k_c5BiacB | eyeseast 25778 | 2022-04-14T19:09:57Z | 2022-04-14T19:09:57Z | CONTRIBUTOR | I wonder if this overlaps with what I outlined in #1605. You could run something like this:
And maybe that does what you need. Of course, that plugin isn't built yet. But that's the idea. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette feature for publishing snapshots of query results 1203943272 | |
1103312860 | https://github.com/simonw/datasette/issues/1713#issuecomment-1103312860 | https://api.github.com/repos/simonw/datasette/issues/1713 | IC_kwDOBm6k_c5Bwzfc | fgregg 536941 | 2022-04-20T00:52:19Z | 2022-04-20T00:52:19Z | CONTRIBUTOR | feels related to #1402 |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette feature for publishing snapshots of query results 1203943272 | |
1111705323 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111705323 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ0br | wragge 127565 | 2022-04-28T03:32:06Z | 2022-04-28T03:32:06Z | CONTRIBUTOR | Ah, that would be it! I have a core set of data which doesn't change to which I want authorised users to be able to submit corrections. I was going to deal with the persistence issue by just grabbing the user corrections at regular intervals and saving to GitHub. I might need to rethink. Thanks! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111712953 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111712953 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ2S5 | wragge 127565 | 2022-04-28T03:48:36Z | 2022-04-28T03:48:36Z | CONTRIBUTOR | I don't think that'd work for this project. The db is very big, and my aim was to have an environment where researchers could be making use of the data, but be easily able to add corrections to the HTR/OCR extracted data when they came across problems. It's in its immutable (!) form here: https://sydney-stock-exchange-xqtkxtd5za-ts.a.run.app/stock_exchange/stocks |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111752676 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111752676 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ__k | wragge 127565 | 2022-04-28T05:11:54Z | 2022-04-28T05:11:54Z | CONTRIBUTOR | And in terms of the bug, yep I agree that option 2 would be the most useful and least frustrating. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1111751734 | https://github.com/simonw/datasette/issues/1728#issuecomment-1111751734 | https://api.github.com/repos/simonw/datasette/issues/1728 | IC_kwDOBm6k_c5CQ_w2 | wragge 127565 | 2022-04-28T05:09:59Z | 2022-04-28T05:09:59Z | CONTRIBUTOR | Thanks, I'll give it a try! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Writable canned queries fail with useless non-error against immutable databases 1218133366 | |
1125342229 | https://github.com/simonw/datasette/issues/741#issuecomment-1125342229 | https://api.github.com/repos/simonw/datasette/issues/741 | IC_kwDOBm6k_c5DE1wV | eyeseast 25778 | 2022-05-12T19:21:16Z | 2022-05-12T19:21:16Z | CONTRIBUTOR | Came here to check if this had been flagged already. Was helping a colleague get something on Cloud Run and had to dig to find If I get some time next week, maybe I'll try to tackle it. Would definitely make things easier to be able to do something like this:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Replace "datasette publish --extra-options" with "--setting" 607223136 | |
1128049716 | https://github.com/simonw/datasette/issues/1742#issuecomment-1128049716 | https://api.github.com/repos/simonw/datasette/issues/1742 | IC_kwDOBm6k_c5DPKw0 | eyeseast 25778 | 2022-05-16T19:24:44Z | 2022-05-16T19:24:44Z | CONTRIBUTOR | Where is |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_trace=1 fails with datasette-geojson for some reason 1237586379 | |
1128064864 | https://github.com/simonw/datasette/issues/1742#issuecomment-1128064864 | https://api.github.com/repos/simonw/datasette/issues/1742 | IC_kwDOBm6k_c5DPOdg | eyeseast 25778 | 2022-05-16T19:42:13Z | 2022-05-16T19:42:13Z | CONTRIBUTOR | Just to add a wrinkle here, this loads fine: https://alltheplaces-datasette.fly.dev/alltheplaces/places.geojson?_trace=1 But also, this doesn't add any trace data: https://alltheplaces-datasette.fly.dev/alltheplaces/places.json?_trace=1 What am I missing? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
?_trace=1 fails with datasette-geojson for some reason 1237586379 | |
1151887842 | https://github.com/simonw/datasette/issues/1528#issuecomment-1151887842 | https://api.github.com/repos/simonw/datasette/issues/1528 | IC_kwDOBm6k_c5EqGni | eyeseast 25778 | 2022-06-10T03:23:08Z | 2022-06-10T03:23:08Z | CONTRIBUTOR | I just put together a version of this in a plugin: https://github.com/eyeseast/datasette-query-files. Happy to have any feedback. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new `"sql_file"` key to Canned Queries in metadata? 1060631257 | |
1173358747 | https://github.com/simonw/datasette/issues/1713#issuecomment-1173358747 | https://api.github.com/repos/simonw/datasette/issues/1713 | IC_kwDOBm6k_c5F8Aib | brandonrobertz 2670795 | 2022-07-04T05:16:35Z | 2022-07-04T05:16:35Z | CONTRIBUTOR | This feature is pretty important and would be nice if it would be all within Datasette (no separate CLI/deploy required). My workflow now is to basically just copy the result and paste into a Google Sheet, which works, but then it's not discoverable to other journalists browsing the Datasette instance. I started building a plugin similar to datasette-saved-queries but one that maintains its own DB (required if you're working with all immutable DBs), but got bogged down in details. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Datasette feature for publishing snapshots of query results 1203943272 | |
1163091750 | https://github.com/simonw/datasette/pull/1753#issuecomment-1163091750 | https://api.github.com/repos/simonw/datasette/issues/1753 | IC_kwDOBm6k_c5FU18m | dependabot[bot] 49699333 | 2022-06-22T13:22:34Z | 2022-06-22T13:22:34Z | CONTRIBUTOR | Superseded by #1760. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump furo from 2022.4.7 to 2022.6.4.1 1261826957 | |
1168704157 | https://github.com/simonw/datasette/pull/1693#issuecomment-1168704157 | https://api.github.com/repos/simonw/datasette/issues/1693 | IC_kwDOBm6k_c5FqQKd | dependabot[bot] 49699333 | 2022-06-28T13:11:36Z | 2022-06-28T13:11:36Z | CONTRIBUTOR | Superseded by #1763. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Bump black from 22.1.0 to 22.3.0 1184850337 | |
1200732975 | https://github.com/simonw/datasette/issues/1191#issuecomment-1200732975 | https://api.github.com/repos/simonw/datasette/issues/1191 | IC_kwDOBm6k_c5Hkbsv | brandonrobertz 2670795 | 2022-08-01T05:39:27Z | 2022-08-01T05:39:27Z | CONTRIBUTOR | I've got a URL shortening plugin that I would like to embed on the query page but I'd like avoid capturing the entire |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Ability for plugins to collaborate when adding extra HTML to blocks in default templates 787098345 | |
1223347322 | https://github.com/simonw/datasette/pull/1789#issuecomment-1223347322 | https://api.github.com/repos/simonw/datasette/issues/1789 | IC_kwDOBm6k_c5I6sx6 | asg017 15178711 | 2022-08-23T00:03:20Z | 2022-08-23T00:03:20Z | CONTRIBUTOR | @simonw to build the extension on ubuntu, you can run:
I'm not the best with Actions, but if you set the cache key to |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new entrypoint option to `--load-extension` 1344823170 | |
1210675046 | https://github.com/simonw/datasette/issues/1779#issuecomment-1210675046 | https://api.github.com/repos/simonw/datasette/issues/1779 | IC_kwDOBm6k_c5IKW9m | fgregg 536941 | 2022-08-10T13:28:37Z | 2022-08-10T13:28:37Z | CONTRIBUTOR | maybe a simpler solution is to set the maxscale to like 2? since datasette is not set up to make use of container scaling anyway? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
google cloudrun updated their limits on maxscale based on memory and cpu count 1334628400 | |
1214437408 | https://github.com/simonw/datasette/issues/1779#issuecomment-1214437408 | https://api.github.com/repos/simonw/datasette/issues/1779 | IC_kwDOBm6k_c5IYtgg | fgregg 536941 | 2022-08-14T19:42:58Z | 2022-08-14T19:42:58Z | CONTRIBUTOR | thanks @simonw! |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
google cloudrun updated their limits on maxscale based on memory and cpu count 1334628400 | |
1221576460 | https://github.com/simonw/datasette/pull/1789#issuecomment-1221576460 | https://api.github.com/repos/simonw/datasette/issues/1789 | IC_kwDOBm6k_c5Iz8cM | asg017 15178711 | 2022-08-21T16:16:42Z | 2022-08-21T16:16:42Z | CONTRIBUTOR | Rebased, Read the docs failure should now now fixed Re docs - ya that's a pretty ambitious page, I'm still not 100% sure what the best practices are/should be... Would be happy to make that page in a future PR |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Add new entrypoint option to `--load-extension` 1344823170 | |
1237381620 | https://github.com/simonw/datasette/pull/1685#issuecomment-1237381620 | https://api.github.com/repos/simonw/datasette/issues/1685 | IC_kwDOBm6k_c5JwPH0 | dependabot[bot] 49699333 | 2022-09-05T18:36:47Z | 2022-09-05T18:36:47Z | CONTRIBUTOR | Looks like jinja2 is no longer updatable, so this is no longer needed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update jinja2 requirement from <3.1.0,>=2.10.3 to >=2.10.3,<3.2.0 1180778860 | |
1237381569 | https://github.com/simonw/datasette/pull/1799#issuecomment-1237381569 | https://api.github.com/repos/simonw/datasette/issues/1799 | IC_kwDOBm6k_c5JwPHB | dependabot[bot] 49699333 | 2022-09-05T18:36:42Z | 2022-09-05T18:36:42Z | CONTRIBUTOR | Looks like aiofiles is no longer updatable, so this is no longer needed. |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Update aiofiles requirement from <0.9,>=0.4 to >=0.4,<22.2 1362242558 | |
1256781274 | https://github.com/simonw/datasette/issues/1817#issuecomment-1256781274 | https://api.github.com/repos/simonw/datasette/issues/1817 | IC_kwDOBm6k_c5K6PXa | jefftriplett 50527 | 2022-09-23T22:59:46Z | 2022-09-23T22:59:46Z | CONTRIBUTOR | While you are adding features, would you be future-proofing your APIs if you switched over some arguments over to keyword-only arguments or would that be too disruptive? Thinking out loud:
|
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Expose `sql` and `params` arguments to various plugin hooks 1384273985 | |
1248204219 | https://github.com/simonw/datasette/issues/1810#issuecomment-1248204219 | https://api.github.com/repos/simonw/datasette/issues/1810 | IC_kwDOBm6k_c5KZhW7 | psychemedia 82988 | 2022-09-15T14:44:47Z | 2022-09-15T14:46:26Z | CONTRIBUTOR | A couple+ of possible use case examples:
In many cases, I suspect the raw content will be in one table, but the search table will be a second (eg FTS) table. Generally, the search may be over one or more joined tables, and the results constructed from one or more tables (which may or may not be distinct from the search tables). |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Featured table(s) on the homepage 1374626873 | |
1258129113 | https://github.com/simonw/datasette/issues/1727#issuecomment-1258129113 | https://api.github.com/repos/simonw/datasette/issues/1727 | IC_kwDOBm6k_c5K_YbZ | fgregg 536941 | 2022-09-26T14:30:11Z | 2022-09-26T14:48:31Z | CONTRIBUTOR | from your analysis, it seems like the GIL is blocking on loading of the data from sqlite to python, (particularly in the this is probably a simplistic idea, but what if you had the python code in the something like: this kind of thing works well with a postgres server side cursor, but i'm not sure if it will hold for sqlite. you would still spend about the same amount of time in python and would be contending for the gil, but it would be could be non blocking. depending on the data flow, this could also some benefit for memory. (data stays in more compact sqlite-land until you need it) |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Research: demonstrate if parallel SQL queries are worthwhile 1217759117 | |
1258167564 | https://github.com/simonw/datasette/issues/526#issuecomment-1258167564 | https://api.github.com/repos/simonw/datasette/issues/526 | IC_kwDOBm6k_c5K_h0M | fgregg 536941 | 2022-09-26T14:57:44Z | 2022-09-26T15:08:36Z | CONTRIBUTOR | reading the database execute method i have a few questions. unless i'm missing something (which is very likely!!), the It's not like adding a Instead the code executes the full original query, and if still has time it fetches out the first this does offer some protection of memory exhaustion, as you won't hydrate a huge result set into python (however, there are data flow patterns that could avoid that too) given the current architecture, i don't see how creating a new connection would be use? If we just removed the |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
Stream all results for arbitrary SQL and canned queries 459882902 | |
1258166572 | https://github.com/simonw/datasette/issues/1655#issuecomment-1258166572 | https://api.github.com/repos/simonw/datasette/issues/1655 | IC_kwDOBm6k_c5K_hks | fgregg 536941 | 2022-09-26T14:57:04Z | 2022-09-26T14:57:04Z | CONTRIBUTOR | I think that paginating, even in javascript, could be very helpful. Maybe render json or csv into the page and let javascript loading that into the dom? |
{ "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
query result page is using 400mb of browser memory 40x size of html page and 400x size of csv data 1163369515 |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE [issue_comments] ( [html_url] TEXT, [issue_url] TEXT, [id] INTEGER PRIMARY KEY, [node_id] TEXT, [user] INTEGER REFERENCES [users]([id]), [created_at] TEXT, [updated_at] TEXT, [author_association] TEXT, [body] TEXT, [reactions] TEXT, [issue] INTEGER REFERENCES [issues]([id]) , [performed_via_github_app] TEXT); CREATE INDEX [idx_issue_comments_issue] ON [issue_comments] ([issue]); CREATE INDEX [idx_issue_comments_user] ON [issue_comments] ([user]);
user >30