github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078343231 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | 1078343231 | IC_kwDOCGYnMM5ARjY_ | 9599 | 2022-03-24T21:16:10Z | 2022-03-24T21:17:20Z | OWNER | Aha! This may be possible already: https://github.com/simonw/sqlite-utils/blob/396f80fcc60da8dd844577114f7920830a2e5403/sqlite_utils/utils.py#L311-L316 And yes, this does indeed work - you can do something like this: ``` echo '{"name": "harry"}' | sqlite-utils insert db.db people - --convert ' import time # Simulate something expensive time.sleep(1) def convert(row): row["upper"] = row["name"].upper() ' ``` And after running that: ``` sqlite-utils dump db.db BEGIN TRANSACTION; CREATE TABLE [people] ( [name] TEXT, [upper] TEXT ); INSERT INTO "people" VALUES('harry','HARRY'); COMMIT; ``` So this is a documentation issue - there's a trick for it but I didn't know what the trick was! | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1178546862 | |
https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078328774 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | 1078328774 | IC_kwDOCGYnMM5ARf3G | 9599 | 2022-03-24T21:12:33Z | 2022-03-24T21:12:33Z | OWNER | Here's how the `_compile_code()` mechanism works at the moment: https://github.com/simonw/sqlite-utils/blob/396f80fcc60da8dd844577114f7920830a2e5403/sqlite_utils/utils.py#L308-L342 At the end it does this: ```python return locals["fn"] ``` So it's already building and then returning a function. The question is if there's a sensible way to allow people to further customize that function by executing some code first, in a way that's easy to explain. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1178546862 | |
https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078322301 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | 1078322301 | IC_kwDOCGYnMM5AReR9 | 9599 | 2022-03-24T21:10:52Z | 2022-03-24T21:10:52Z | OWNER | I can think of three ways forward: - Figure out a pattern that gets that local file import workaround to work - Add another option such as `--convert-init` that lets you pass code that will be executed once at the start - Come up with a pattern where the `--convert` code can run some initialization code and then return a function which will be called against each value I quite like the idea of that third option - I'm going to prototype it and see if I can work something out. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1178546862 | |
https://github.com/simonw/sqlite-utils/issues/420#issuecomment-1078315922 | https://api.github.com/repos/simonw/sqlite-utils/issues/420 | 1078315922 | IC_kwDOCGYnMM5ARcuS | 9599 | 2022-03-24T21:09:27Z | 2022-03-24T21:09:27Z | OWNER | Yeah, this is WAY harder than it should be. There's a clumsy workaround you could use which looks something like this: create a file `my_enchant.py` containing: ```python import enchant d = enchant.Dict("en_US") def check(word): return d.check(word) ``` Then run `sqlite-utils` like this: ``` PYTHONPATH=. cat items.json | jq '.data' | sqlite-utils insert listings.db listings - --convert 'my_enchant.check(value)' --import my_enchant ``` Except I tried that and it doesn't work! I don't know the right pattern for getting `--import` to work with modules in the same directory. So yeah, this is definitely a big feature gap. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
1178546862 |