issue_comments

https://sitesforprofit.com/responsive-table-plugins-and-patterns has some useful links.

I really like the pattern from https://css-tricks.com/responsive-data-tables/

/* 
Max width before this PARTICULAR table gets nasty
This query will take effect for any screen smaller than 760px
and also iPads specifically.
*/
@media 
only screen and (max-width: 760px),
(min-device-width: 768px) and (max-device-width: 1024px)  {

    /* Force table to not be like tables anymore */
    table, thead, tbody, th, td, tr { 
        display: block; 
    }

    /* Hide table headers (but not display: none;, for accessibility) */
    thead tr { 
        position: absolute;
        top: -9999px;
        left: -9999px;
    }

    tr { border: 1px solid #ccc; }

    td { 
        /* Behave  like a "row" */
        border: none;
        border-bottom: 1px solid #eee; 
        position: relative;
        padding-left: 50%; 
    }

    td:before { 
        /* Now like a table header */
        position: absolute;
        /* Top/left values mimic padding */
        top: 6px;
        left: 6px;
        width: 45%; 
        padding-right: 10px; 
        white-space: nowrap;
    }

    /*
    Label the data
    */
    td:nth-of-type(1):before { content: "First Name"; }
    td:nth-of-type(2):before { content: "Last Name"; }
    td:nth-of-type(3):before { content: "Job Title"; }
    td:nth-of-type(4):before { content: "Favorite Color"; }
    td:nth-of-type(5):before { content: "Wars of Trek?"; }
    td:nth-of-type(6):before { content: "Porn Name"; }
    td:nth-of-type(7):before { content: "Date of Birth"; }
    td:nth-of-type(8):before { content: "Dream Vacation City"; }
    td:nth-of-type(9):before { content: "GPA"; }
    td:nth-of-type(10):before { content: "Arbitrary Data"; }
}

Could I use https://sqlparse.readthedocs.io/en/latest/ to parse incoming statements and ensure they are pure SELECTs? Would that prevent people from using a compound SELECT statement to trigger an evil PRAGMA of some sort?

It certainly looks like some of the stuff in https://sqlite.org/pragma.html could be used to screw around with things. Example: PRAGMA case_sensitive_like = 1 - would that affect future queries?

Simplest implementation will be to create a temporary directory somewhere, copy in a Dockerfile and the databases and run “now” in it.

Ideally I can use symlinks rather than copying potentially large database files around.

Ideally we can get some serious gains from the fact that our database file is opened with the immutable option.

The gold standard here is to be able to serve up increasingly large datasets without blocking the event loop and while using a sustainable amount of RAM

If this does work, I need to figure it what to do about the HTML view. ASsuming I can iteratively produce JSON and CSV, what to do about HTML? One option: render the first 500 rows as HTML, then hand off to an infinite scroll experience that iteratively loads more rows as JSON.

First experiment: hook up an iterative CSV dump (just because that’s a tiny bit easier to get started with than iterative a JSON). Have it execute a big select statement and then iterate through the result set 100 rows at a time using sqite fetchmany() - also have it async sleep for a second in between each batch of 100.

Can this work without needing python threads?

Could this be as simple as using the iterative JSON encoder and adding a yield statement in between each chunk?

I had to manually set the content disposition header:

return await response.file_stream(
    filepath, headers={
        'Content-Disposition': 'attachment; filename="{}"'.format(ilepath)
    }
)

In the next release of Sanic I can just use the filename= argument instead:

https://github.com/channelcat/sanic/commit/07e95dba4f5983afc1e673df14bdd278817288aa

I'm going to call this one done for the moment. The date filters can go in a stretch goal.

Still to do:

• [x] gt, gte, lt, lte
• [x] like
• [x] glob

/database-name/table-name?name__contains=simon&sort=id+desc

Note that if there's a column called "sort" you can still do sort__exact=blah