github
html_url | issue_url | id | node_id | user | created_at | updated_at | author_association | body | reactions | issue | performed_via_github_app |
---|---|---|---|---|---|---|---|---|---|---|---|
https://github.com/simonw/sqlite-utils/issues/182#issuecomment-701626134 | https://api.github.com/repos/simonw/sqlite-utils/issues/182 | 701626134 | MDEyOklzc3VlQ29tbWVudDcwMTYyNjEzNA== | 9599 | 2020-09-30T20:27:09Z | 2020-09-30T20:27:42Z | OWNER | It looks like http://maps.natalian.org/data.txt is encoded as `latin-1`, but `sqlite-utils` assumes `utf-8` and hence breaks. It would be worth improving the error message here. I could also add a `--encoding latin-1` option to `sqlite-utils insert` to help in consuming files that are stored in charsets other than `utf-8`. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
711649325 | |
https://github.com/simonw/sqlite-utils/issues/182#issuecomment-708693133 | https://api.github.com/repos/simonw/sqlite-utils/issues/182 | 708693133 | MDEyOklzc3VlQ29tbWVudDcwODY5MzEzMw== | 9599 | 2020-10-14T22:26:26Z | 2020-10-14T22:26:26Z | OWNER | I could use https://github.com/chardet/chardet to help here, though I'd rather not add it as a dependency (`sqlite-utils` has very few dependencies at the moment). I could add it as an optional dependency though. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
711649325 | |
https://github.com/simonw/sqlite-utils/issues/182#issuecomment-708695761 | https://api.github.com/repos/simonw/sqlite-utils/issues/182 | 708695761 | MDEyOklzc3VlQ29tbWVudDcwODY5NTc2MQ== | 9599 | 2020-10-14T22:33:55Z | 2020-10-14T22:35:17Z | OWNER | For the moment I'm going to add a `--encoding` option and some code that catches `UnicodeDecodeError` and shows an error message that suggests using `--encoding`. That error message could detect if the `file` command is available and, if it is, suggest running `file filename.txt` to detect the character encoding. | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
711649325 | |
https://github.com/simonw/sqlite-utils/issues/182#issuecomment-710178871 | https://api.github.com/repos/simonw/sqlite-utils/issues/182 | 710178871 | MDEyOklzc3VlQ29tbWVudDcxMDE3ODg3MQ== | 9599 | 2020-10-16T16:27:39Z | 2020-10-16T16:28:14Z | OWNER | The file is opened for me by `click.File()`, which also handles things like `-` for stdin. But i neee to be able to switch the encoding used to read from that based on the `--encoding` option. I think the way to do that is to open the file in binary mode and then wrap it in a codec reader: ```python fp = codecs.getreader(encoding)(fp) ``` | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
711649325 | |
https://github.com/simonw/sqlite-utils/issues/182#issuecomment-710258736 | https://api.github.com/repos/simonw/sqlite-utils/issues/182 | 710258736 | MDEyOklzc3VlQ29tbWVudDcxMDI1ODczNg== | 9599 | 2020-10-16T17:20:41Z | 2020-10-16T17:20:41Z | OWNER | Documentation: https://sqlite-utils.readthedocs.io/en/latest/cli.html#inserting-csv-or-tsv-data | { "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 } |
711649325 |