issues

Table actions

2 rows where user = 770231 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

state 2

repo 2

type 1

id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1178546862 I_kwDOCGYnMM5GPzKu 420 Document how to use a `--convert` function that runs initialization code first strada 770231 closed 0     12 2022-03-23T19:07:36Z 2022-08-28T11:34:37Z 2022-03-25T20:07:33Z NONE  

When I have an insert command with transform like this:

cat items.json | jq '.data' | sqlite-utils insert listings.db listings - --convert ' d = enchant.Dict("en_US") row["is_dictionary_word"] = d.check(row["name"]) ' --import=enchant --ignore

I noticed as the number of rows increases the operation becomes quite slow, likely due to the creation of the d = enchant.Dict("en_US") object for each row. Is there a way to share that instance d between transform function calls, like a shared context?

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/420/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
1121121305 I_kwDOBm6k_c5C0vQZ 1618 Reconsider policy on blocking queries containing the string "pragma" strada 770231 open 0     6 2022-02-01T19:39:46Z 2022-02-02T19:42:03Z   NONE  

First of all, thanks for creating this cool project, and also supporting publishing to various hosting services out of the box.

While testing out, I noticed legitimate queries such as select * from books where title like 'Pragmatic%' or select * from books where title = 'The Pragmatic Programmer' are blocked, due to the regular expression check here: https://github.com/simonw/datasette/blob/main/datasette/utils/init.py#L185

Example as seen from a Datasette instance: https://fivethirtyeight.datasettes.com/polls?sql=select+*+from+books+where+title+like+%27Pragmatic%25%27%0D%0A

I'd propose a regular expression like re.compile(f"pragma_(?!({'|'.join(allowed_pragmas)}))"), instead of re.compile(f"pragma(?!_({'|'.join(allowed_pragmas)}))"),

I can create a pull request with this change, unless the maintainers think it would allow unwanted queries to be executed.

 datasette 107914493 issue     
{
    "url": "https://api.github.com/repos/simonw/datasette/issues/1618/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
    

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);