Original title: Generating URL for a row inside render_cell hook**

Hey, I am using Datasette to view a database that contains video metadata. It has BLOB columns that contain video thumbnails in JPG format (around 100-500KB per row).

I've registered an output formatter that extends datasette.blob_renderer.render_blob function and serves the column with image/jpeg content type.

```python from datasette.blob_renderer import render_blob

async def render_jpg(datasette, database, rows, columns, request, table, view_name): response = await render_blob(datasette, database, rows, columns, request, table, view_name) response.content_type = "image/jpeg" response.headers["Content-Disposition"] = f'inline; filename="image.jpg"' return response

@hookimpl def register_output_renderer(): return { "extension": "jpg", "render": render_jpg, "can_render": lambda: True, } ```

This works well. I can visit http://localhost:8001/mydb/videos/1.jpg?_blob_column=thumbnail and view the image.

I want to display the image directly with an <img> tag (lazy-loaded of course). So, I need a URL, because embedding base64 would increase the page size too much (each image > 100KB).

Datasette generates a link with .blob extension for blob columns. It does this by calling datasette.urls.row_blob

https://github.com/simonw/datasette/blob/7a2ed9f8a119e220b66d67c7b9e07cbab47b1196/datasette/views/table.py#L169-L179

But I have no way of getting the row inside the render_cell hook.

python @hookimpl def render_cell(value, column, table, database, datasette): if isinstance(value, bytes) and imghdr.what(None, value): # generate url return '$renderedLink'

Any pointers?

Hey, I have a database that I save in HTML from couple of web scrapers. There are around 200k+, 50+ rows in a couple of tables, with sqlite file weighing around 600MB.

The app runs on a VPS with 2 core CPU, 4GB RAM and refreshing database page regularly takes more than 10 seconds. I was suspecting that counting tables was the culprit, but manually running select count(*) from table_name for the largest table finishes under a second.

I've looked at the source code. There's a check for index page for mutable databases larger than 100MB https://github.com/simonw/datasette/blob/799c5d53570d773203527f19530cf772dc2eeb24/datasette/views/index.py#L15

but this check is not performed for database page. I've manually crippled Database::table_counts method py async def table_counts(self, limit=10): if not self.is_mutable and self.cached_table_counts is not None: return self.cached_table_counts # Try to get counts for each table, $limit timeout for each count counts = {} for table in await self.table_names(): try: # table_count = ( # await self.execute( # "select count(*) from [{}]".format(table), # custom_time_limit=limit, # ) # ).rows[0][0] counts[table] = 10 # table_count # In some cases I saw "SQL Logic Error" here in addition to # QueryInterrupted - so we catch that too: except (QueryInterrupted, sqlite3.OperationalError, sqlite3.DatabaseError): counts[table] = None if not self.is_mutable: self.cached_table_counts = counts return counts

now the page loads in <100ms.

Is it possible to apply size check on database page too?

{
    "python": {
        "version": "3.8.0",
        "full": "3.8.0 (default, Oct 28 2019, 16:14:01) \n[GCC 8.3.0]"
    },
    "datasette": {
        "version": "0.44"
    },
    "asgi": "3.0",
    "uvicorn": "0.11.5",
    "sqlite": {
        "version": "3.22.0",
        "fts_versions": [
            "FTS5",
            "FTS4",
            "FTS3"
        ],
        "extensions": {
            "json1": null
        },
        "compile_options": [
            "COMPILER=gcc-7.4.0",
            "ENABLE_COLUMN_METADATA",
            "ENABLE_DBSTAT_VTAB",
            "ENABLE_FTS3",
            "ENABLE_FTS3_PARENTHESIS",
            "ENABLE_FTS3_TOKENIZER",
            "ENABLE_FTS4",
            "ENABLE_FTS5",
            "ENABLE_JSON1",
            "ENABLE_LOAD_EXTENSION",
            "ENABLE_PREUPDATE_HOOK",
            "ENABLE_RTREE",
            "ENABLE_SESSION",
            "ENABLE_STMTVTAB",
            "ENABLE_UNLOCK_NOTIFY",
            "ENABLE_UPDATE_DELETE_LIMIT",
            "HAVE_ISNAN",
            "LIKE_DOESNT_MATCH_BLOBS",
            "MAX_SCHEMA_RETRY=25",
            "MAX_VARIABLE_NUMBER=250000",
            "OMIT_LOOKASIDE",
            "SECURE_DELETE",
            "SOUNDEX",
            "TEMP_STORE=1",
            "THREADSAFE=1"
        ]
    }
}

Fixes https://github.com/simonw/datasette/issues/1129

Fixes https://github.com/simonw/datasette/issues/1094

This import isn't used at all, and causes error on startup on Windows.

Footer doesn't stick to the bottom if the body content isn't long enough to reach the end of viewport.

This can be fixed using flexbox.

```css body { min-height: 100vh; display: flex; flex-direction: column; }

.content { flex-grow: 1; } ```

Fixes https://github.com/simonw/datasette/issues/1121

At the moment it looks like this https://datasette-graphql-demo.datasette.io/github/repos

Adding a few flex statements fixes the alignment and centers h1 text and the cog icon vertically.

Just a quick heads up:

Release notes for 0.45 include urls that point to localhost.

https://github.com/simonw/datasette/releases/tag/0.45

Potential fix for https://github.com/simonw/datasette/issues/859.

Disabling table counts for hidden tables speeds up database page quite a bit. In my setup it reduced load time by 2/3 (~300 -> ~90ms)

Hey,

I'm trying to get canned inserts to work. I have an DB with following metadata:

```text sqlite> .mode line

sqlite> select name, sql from sqlite_master where name like '%search%'; name = search sql = CREATE TABLE "search" ("id" INTEGER NOT NULL PRIMARY KEY, "name" VARCHAR(255) NOT NULL, "url" VARCHAR(255) NOT NULL) ```

```yaml

...

queries: add_search: sql: insert into search(name, url) VALUES (:name, :url), write: true `` which renders a form as expected, but when I submit the form I getincomplete input` error.

but when submit post the form

I've attached a debugger to see where the error comes from, because incomplete input string doesn't appear in datasette codebase.

Inside datasette.database.Database.execute_write_fn

https://github.com/simonw/datasette/blob/4fa7cf68536628344356d3ef8c92c25c249067a0/datasette/database.py#L69

py result = await reply_queue.async_q.get()

this line raises an exception.

That led me to believe I had something wrong with my SQL. But running the command in sqlite3 inserts the record just fine.

text sqlite> insert into search (name, url) values ('my name', 'my url'); sqlite> SELECT last_insert_rowid(); last_insert_rowid() = 3

So I'm a bit lost here.

• datasette, version 0.44
• Python 3.8.1

While debugging why my static mounts using a relative path (--static mystatic:rel/path/to/dir) not working, I noticed that the requests fail no matter what, returning 404 errors.

The reason is that datasette tries to prevent traversal exploits by checking if the path is relative to its registered directory. This check fails when the mount is a relative directory, because /abs/dir/file obviously not under dir/file.

https://github.com/simonw/datasette/blob/81fa8b6cdc5457b42a224779e5291952314e8d20/datasette/utils/asgi.py#L303-L306

This also has the consequence of returning any requested file, because when /abs/dir/../../evil.file resolves aiofiles happily returns it to the client after it resolves the path itself. The solution is to make sure we're checking relativity of paths after they're fully resolved.

I've implemented the mentioned changes and also updated the tests.

Datasette fails to serve files from static mounts that are created using relative paths datasette --static mystatic:rel/path/to/static/dir.
I've explained the problem and the solution in the pull request: https://github.com/simonw/datasette/pull/554

This PR allows running datasette using python -m datasette command in addition to just running the executable.

This function is quite useful when debugging a plugin in a project because IDEs like PyCharm can easily start a debug session when datasette is run as a module in contrast to trying to attach a debugger to a running process.