issues

Table actions

3 rows where comments = 15, repo = 140912432 and user = 9599 sorted by updated_at descending

✎ View and edit SQL

This data as json, CSV (advanced)

Suggested facets: created_at (date), updated_at (date), closed_at (date)

type 2

state 1

repo 1

  • sqlite-utils · 3 ✖
id node_id number title user state locked assignee milestone comments created_at updated_at ▲ closed_at author_association pull_request body repo type active_lock_reason performed_via_github_app reactions draft state_reason
1094890366 PR_kwDOCGYnMM4wlm3B 361 --lines and --text and --convert and --import simonw 9599 closed 0     15 2022-01-06T01:49:44Z 2022-01-06T06:37:03Z 2022-01-06T06:24:54Z OWNER simonw/sqlite-utils/pulls/361

Refs #356

Still TODO:

  • [x] Get --lines working, with tests
  • [x] Get --text working, with tests
  • [x] Get regular JSON import working with --convert with tests
  • [x] Get --lines working with --convert with tests
  • [x] Get --text working with --convert with tests
  • [x] Get --csv and --tsv import working with --convert with tests
  • [x] Get --nl working with --convert with tests
  • [x] Documentation for all of the above
 sqlite-utils 140912432 pull     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/361/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
 0  
841377702 MDU6SXNzdWU4NDEzNzc3MDI= 251 "sqlite-utils convert" command to replace the separate "sqlite-transform" tool simonw 9599 closed 0     15 2021-03-25T22:36:36Z 2021-08-02T22:39:46Z 2021-08-02T04:47:40Z OWNER  

See https://github.com/simonw/sqlite-transform/issues/11 - I built a separate sqlite-transform tool a while ago that uses the word "transform" to means something entirely different from sqlite-utils transform - I'd like to resolve this by merging the two tools.

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/251/reactions",
    "total_count": 0,
    "+1": 0,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed
521868864 MDU6SXNzdWU1MjE4Njg4NjQ= 66 The ".upsert()" method is misnamed simonw 9599 closed 0     15 2019-11-12T23:48:28Z 2019-12-31T01:30:21Z 2019-12-31T01:30:20Z OWNER  

This thread here is illuminating: https://stackoverflow.com/questions/3634984/insert-if-not-exists-else-update

The term UPSERT in SQLite has a specific meaning as-of 3.24.0 (2018-06-04): https://www.sqlite.org/lang_UPSERT.html

It means "behave as an UPDATE or a no-op if the INSERT would violate a uniqueness constraint". The syntax in 3.24.0+ looks like this (confusingly it does not use the term "upsert"): sql INSERT INTO phonebook(name,phonenumber) VALUES('Alice','704-555-1212') ON CONFLICT(name) DO UPDATE SET phonenumber=excluded.phonenumber Here's the problem: the sqlite-utils .upsert() and .upsert_all() methods don't do this. They use the following SQL:

sql INSERT OR REPLACE INTO [{table}] ({columns}) VALUES {rows};

If the record already exists, it will be entirely replaced by a new record - as opposed to updating any specified fields but leaving existing fields as they are (the behaviour of "upsert" in SQLite itself).

 sqlite-utils 140912432 issue     
{
    "url": "https://api.github.com/repos/simonw/sqlite-utils/issues/66/reactions",
    "total_count": 1,
    "+1": 1,
    "-1": 0,
    "laugh": 0,
    "hooray": 0,
    "confused": 0,
    "heart": 0,
    "rocket": 0,
    "eyes": 0
}
   completed

Advanced export

JSON shape: default, array, newline-delimited, object

CSV options: 

CREATE TABLE [issues] (
   [id] INTEGER PRIMARY KEY,
   [node_id] TEXT,
   [number] INTEGER,
   [title] TEXT,
   [user] INTEGER REFERENCES [users]([id]),
   [state] TEXT,
   [locked] INTEGER,
   [assignee] INTEGER REFERENCES [users]([id]),
   [milestone] INTEGER REFERENCES [milestones]([id]),
   [comments] INTEGER,
   [created_at] TEXT,
   [updated_at] TEXT,
   [closed_at] TEXT,
   [author_association] TEXT,
   [pull_request] TEXT,
   [body] TEXT,
   [repo] INTEGER REFERENCES [repos]([id]),
   [type] TEXT
, [active_lock_reason] TEXT, [performed_via_github_app] TEXT, [reactions] TEXT, [draft] INTEGER, [state_reason] TEXT);
CREATE INDEX [idx_issues_repo]
                ON [issues] ([repo]);
CREATE INDEX [idx_issues_milestone]
                ON [issues] ([milestone]);
CREATE INDEX [idx_issues_assignee]
                ON [issues] ([assignee]);
CREATE INDEX [idx_issues_user]
                ON [issues] ([user]);