github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/7#issuecomment-458011906 | https://api.github.com/repos/simonw/sqlite-utils/issues/7 | 458011906 | MDEyOklzc3VlQ29tbWVudDQ1ODAxMTkwNg== | 9599 | 2019-01-28T06:25:55Z | 2019-01-28T06:25:55Z | OWNER | I tested this with a script called `churn_em_out.py` ``` i = 0 while True: i += 1 print( '{"id": I, "another": "row", "number": J}'.replace("I", str(i)).replace( "J", str(i + 1) ) ) ``` Then I ran this: ``` python churn_em_out.py | \ sqlite-utils insert /tmp/getbig.db stats - \ --nl --batch-size=10000 ``` And used `watch 'ls -lah /tmp/getbig.db'` to watch the file growing as it had 10,000 lines of junk committed in batches. The memory used by the process never grew about around 50MB. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
403625674 | |
https://github.com/simonw/sqlite-utils/issues/7#issuecomment-458011885 | https://api.github.com/repos/simonw/sqlite-utils/issues/7 | 458011885 | MDEyOklzc3VlQ29tbWVudDQ1ODAxMTg4NQ== | 9599 | 2019-01-28T06:25:48Z | 2019-01-28T06:25:48Z | OWNER | Re-opening for the second bit involving the cli tool. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
403625674 | |
https://github.com/simonw/sqlite-utils/issues/7#issuecomment-457980966 | https://api.github.com/repos/simonw/sqlite-utils/issues/7 | 457980966 | MDEyOklzc3VlQ29tbWVudDQ1Nzk4MDk2Ng== | 9599 | 2019-01-28T02:29:32Z | 2019-01-28T02:29:32Z | OWNER | Remember to remove this TODO (and turn the `[]` into `()` on this line) as part of this task: https://github.com/simonw/sqlite-utils/blob/5309c5c7755818323a0f5353bad0de98ecc866be/sqlite_utils/cli.py#L78-L80 | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
403625674 |