issue_comments

I shipped 2.0 - release notes here: https://sqlite-utils.readthedocs.io/en/stable/changelog.html#v2

I also wrote about it on my blog: https://simonwillison.net/2019/Dec/30/sqlite-utils-2/

Last step: update changelog and ship 2.0. Then I can close this issue.

I'm going to start by ignoring the existing upsert entirely and implementing .insert(..., replace=True) and $ sqlite-utils insert --replace. Including updating the tests.

Then I'll figure out how to implement the new .upsert() / $ sqlite-utils upsert.

Then I'll update the documentation, and ship sqlite-utils 2.0.

Don't forget to update the documentation. This will be quite an involved task.

Thinking about this further: I believe every time I've personally used upsert in the past (either with the Python library or the CLI tool) I've actually wanted the new behaviour, where "upsert" means "update existing record with these changes, or insert a new record if one does not exist".

So I'm happy with upsert doing that, and insert --replace being added as an option that does what upsert does ta the moment.

I'll still ship it as version 2.0 since it's technically a breaking change.

Urgh this is going to be quite a bit of work, especially in the CLI module which shares an implementation for upsert and insert in a way that looks like it will have to be unwrapped.

First step: add a replace=True argument to insert() and insert_all() that does the same thing as the current upsert=True

https://github.com/simonw/sqlite-utils/blob/8dab9fd1ccf571e188eec9ccf606a0c50fccf200/sqlite_utils/db.py#L938-L946

Maybe instead of inventing a new term I should tell people to use .insert(..., replace=True) directly. That matches ignore=True.

This is going to affect the design of the CLI subcommands as well.

Maybe inplace() (combining "insert" and "replace")?

It could be an alias for .insert(..., replace=True)

Is replace() a good name here? It doesn't really convey the idea that a brand new record will be created if there isn't an existing one to replace.

This warrants making a backwards compatible change, which means I'll need to bump the major version number and release 2.0.

I'm going to rename the existing upsert() and upsert_all() methods to replace() and replace_all() - then write new upsert() and upsert_all() methods that implement the correct behavior.

If I do implement the correct definition of .upsert() I think I'll use this pattern, since it works in versions of SQLite prior to 3.24: sql INSERT OR IGNORE INTO book(id) VALUES(1001); UPDATE book SET name = 'Programming' WHERE id = 1001;

Fixing this is going to be a real pain. There's lots of code out there that uses sqlite-utils with the expectation that upsert() behaves as it currently does.

Maybe I need to introduce new terms for both of these different patterns and deprecate the existing .upsert() and .upsert_all() since their behaviour can't be changed?

Or maybe I fix this and ship sqlite-utils 2.0 with a breaking change?

This relates to this bug: https://github.com/dogsheep/github-to-sqlite/pull/8#issuecomment-549233778