issue_comments

Thanks for the reply @simonw. What services have you had better success with than Cloud Run for larger database?

Also, what about my issue description makes you think there may be a workaround?

Is there any instrumentation I could add to see at which point in the deploy the memory usage spikes? Should I be able to see this whether it's running under Docker locally, or do you suspect this is Cloud Run-specific?

The 3.10 tests failed a lot. Trying to run this locally:

``` /tmp % pyenv install 3.10 python-build: definition not found: 3.10

The following versions contain `3.10' in the name: 3.10.0a6 3.10-dev miniconda-3.10.1 miniconda3-3.10.1

See all available versions with `pyenv install --list'.

If the version you need is missing, try upgrading pyenv:

brew update && brew upgrade pyenv ``` So trying:

brew update && brew upgrade pyenv

Then did this:

/tmp % brew upgrade pyenv ==> Upgrading 1 outdated package: pyenv 1.2.24.1 -> 2.1.0 This decided to upgrade everything by downloaded everything on the internet. Aah, Homebrew.

But it looks like I have 3.10.0 available to pyenv now.

/tmp % pyenv install 3.10.0 python-build: use openssl@1.1 from homebrew python-build: use readline from homebrew Downloading Python-3.10.0.tar.xz... -> https://www.python.org/ftp/python/3.10.0/Python-3.10.0.tar.xz Installing Python-3.10.0... ...

I've had this problem too - my solution was to not use Cloud Run for databases larger than about 2GB, but the way you describe it here makes me think that maybe there is a workaround here which could get it to work.

On further investigation this isn't related to _search at all - it happens when you explicitly sort by _sort=rowid and apply a _next

• https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200 works without an error (currently)
• https://global-power-plants.datasettes.com/global-power-plants/global-power-plants?_next=200&_sort=rowid shows that error

Here's the exception: -> params[f"p{len(params)}"] = components[0] (Pdb) list 603 604 # Figure out the SQL for next-based-on-primary-key first 605 next_by_pk_clauses = [] 606 if use_rowid: 607 next_by_pk_clauses.append(f"rowid > :p{len(params)}") 608 -> params[f"p{len(params)}"] = components[0] 609 else: 610 # Apply the tie-breaker based on primary keys 611 if len(components) == len(pks): 612 param_len = len(params) 613 next_by_pk_clauses.append( Debugger shows that components is an empty array, so components[0] cannot be resolved:

-> params[f"p{len(params)}"] = components[0] (Pdb) params {'search': 'hello'} (Pdb) components []

So the bug is in this code: https://github.com/simonw/datasette/blob/adb5b70de5cec3c3dd37184defe606a082c232cf/datasette/views/table.py#L604-L617