https://github.com/simonw/datasette-cors seems to workaround this

if you went this route:

python with sqlite_timelimit(conn, time_limit_ms): c.execute(query) for chunk in c.fetchmany(chunk_size): yield from chunk

then time_limit_ms would probably have to be greatly extended, because the time spent in the loop will depend on the downstream processing.

i wonder if this was why you were thinking this feature would need a dedicated connection?

reading more, there's no real limit i can find on the number of active cursors (or more precisely active prepared statements objects, because sqlite doesn't really have cursors).

maybe something like this would be okay?

python with sqlite_timelimit(conn, time_limit_ms): c.execute(query) # step through at least one to evaluate the statement, not sure if this is necessary yield c.execute.fetchone() for chunk in c.fetchmany(chunk_size): yield from chunk

this seems quite weird that there's not more of limit of the number of active prepared statements, but i haven't been able to find one.

for teaching register_output_renderer to stream it seems like the two options are to

1. a nested query technique to paginate through
2. a fetching model that looks like something python with sqlite_timelimit(conn, time_limit_ms): c.execute(query) for chunk in c.fetchmany(chunk_size): yield from chunk currently db.execute is not a generator, so this would probably need a new method?

I'm glad the bug report served some purpose. Frankly I just needed the method signature, that is why the documentation you mention wasn't read.

On Tue, Sep 27, 2022, 9:05 PM Simon Willison @.***> wrote:

Though now I notice that the copy right there needs to be updated to reflect the new row parameter to render_cell!

— Reply to this email directly, view it on GitHub https://github.com/simonw/datasette/issues/1826#issuecomment-1260357878, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP46PCLI6KVDQTFLWVZODRLWAO72JANCNFSM6AAAAAAQXKWIJA . You are receiving this because you authored the thread.Message ID: @.***>

This is great, thank you very much!

https://datasette--1825.org.readthedocs.build/en/1825/deploying.html#running-datasette-using-openrc

Codecov Report

Base: 91.58% // Head: 91.58% // No change to project coverage :thumbsup:

Coverage data is based on head (b16eb2f) compared to base (5f9f567). Patch has no changes to coverable lines.

:exclamation: Current head b16eb2f differs from pull request most recent head e7e96dc. Consider uploading reports for the commit e7e96dc to get more accurate results

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

Though now I notice that the copy right there needs to be updated to reflect the new row parameter to render_cell!

This is deliberate. The Datasette plugin system allows you to specify only a subset of the parameters for a hook - in this example, only the value is needed so the others can be omitted.

There's a note about this at the very top of that documentation page: https://docs.datasette.io/en/stable/plugin_hooks.html#plugin-hooks

When you implement a plugin hook you can accept any or all of the parameters that are documented as being passed to that hook.

For example, you can implement the render_cell plugin hook like this even though the full documented hook signature is render_cell(value, column, table, database, datasette): python @hookimpl def render_cell(value, column): if column == "stars": return "*" * int(value)

The ultimate protection against those memory bombs is to support more streaming output formats. Related issues:

• 1177

• 1062