{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-1001115286", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 1001115286, "node_id": "IC_kwDOCGYnMM47q86W", "user": {"value": 1206106, "label": "agguser"}, "created_at": "2021-12-26T07:01:31Z", "updated_at": "2021-12-26T07:01:31Z", "author_association": "NONE", "body": "`--no-headers` does not work?\r\n```\r\n$ echo 'a,1\\nb,2' | sqlite-utils memory --no-headers -t - 'select * from stdin'\r\na      1                                                                                                                             \r\n---  ---                                                                                                                             \r\nb      2 \r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778851721", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778851721, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODg1MTcyMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-14T22:23:46Z", "updated_at": "2021-02-14T22:23:46Z", "author_association": "OWNER", "body": "I called this `--no-headers` for consistency with the existing output option: https://github.com/simonw/sqlite-utils/blob/427dace184c7da57f4a04df07b1e84cdae3261e8/sqlite_utils/cli.py#L61-L64", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778849394", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778849394, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODg0OTM5NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-14T22:06:53Z", "updated_at": "2021-02-14T22:06:53Z", "author_association": "OWNER", "body": "For the moment I think just adding `--no-header` - which causes column names \"unknown1,unknown2,...\" to be used - should be enough.\r\n\r\nUsers can import with that option, then use `sqlite-utils transform --rename` to rename them.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778811746", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778811746, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODgxMTc0Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-14T17:39:30Z", "updated_at": "2021-02-14T21:16:54Z", "author_association": "OWNER", "body": "I'm going to detach this from the #131 column types idea.\r\n\r\nThe three things I need to handle here are:\r\n\r\n- The CSV file doesn't have a header row at all, so I need to specify what the column names should be\r\n- The CSV file DOES have a header row but I want to ignore it and use alternative column names\r\n- The CSV doesn't have a header row at all and I want to automatically use `unknown1,unknown2...` so I can start exploring it as quickly as possible.\r\n\r\nHere's a potential design that covers the first two:\r\n\r\n`--replace-header=\"foo,bar,baz\"` - ignore whatever is in the first row and pretend it was this instead\r\n`--add-header=\"foo,bar,baz\"` - add a first row with these details, to use as the header\r\n\r\nIt doesn't cover the \"give me unknown column names\" case though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778843086", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778843086, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODg0MzA4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-14T21:15:43Z", "updated_at": "2021-02-14T21:15:43Z", "author_association": "OWNER", "body": "I'm not convinced the `.has_header()` rules are useful for the kind of CSV files I work with: https://github.com/python/cpython/blob/63298930fb531ba2bb4f23bc3b915dbf1e17e9e1/Lib/csv.py#L383\r\n\r\n```python\r\n    def has_header(self, sample):\r\n        # Creates a dictionary of types of data in each column. If any\r\n        # column is of a single type (say, integers), *except* for the first\r\n        # row, then the first row is presumed to be labels. If the type\r\n        # can't be determined, it is assumed to be a string in which case\r\n        # the length of the string is the determining factor: if all of the\r\n        # rows except for the first are the same length, it's a header.\r\n        # Finally, a 'vote' is taken at the end for each column, adding or\r\n        # subtracting from the likelihood of the first row being a header.\r\n```\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778842982", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778842982, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODg0Mjk4Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-14T21:15:11Z", "updated_at": "2021-02-14T21:15:11Z", "author_association": "OWNER", "body": "Implementation tip: I have code that reads the first row and uses it as headers here: https://github.com/simonw/sqlite-utils/blob/8f042ae1fd323995d966a94e8e6df85cc843b938/sqlite_utils/cli.py#L689-L691\r\n\r\nSo If I want to use `unknown1,unknown2...` I can do that by reading the first row, counting the number of columns, generating headers based on that range and then continuing to build that generator (maybe with `itertools.chain()` to replay the record we already read).\r\n\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778812050", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778812050, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODgxMjA1MA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-14T17:41:30Z", "updated_at": "2021-02-14T17:41:30Z", "author_association": "OWNER", "body": "I just spotted that `csv.Sniffer` in the Python standard library has a `.has_header(sample)` method which detects if the first row appears to be a header or not, which is interesting. https://docs.python.org/3/library/csv.html#csv.Sniffer", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778811934", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778811934, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODgxMTkzNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-14T17:40:48Z", "updated_at": "2021-02-14T17:40:48Z", "author_association": "OWNER", "body": "Another pattern that might be useful is to generate a header that is just \"unknown1,unknown2,unknown3\" for each of the columns in the rest of the file. This makes it easy to e.g. facet-explore within Datasette to figure out the correct names, then use `sqlite-utils transform --rename` to rename the columns.\r\n\r\nI needed to do that for the https://bl.iro.bl.uk/work/ns/3037474a-761c-456d-a00c-9ef3c6773f4c example.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778511347", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778511347, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODUxMTM0Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-12T23:27:50Z", "updated_at": "2021-02-12T23:27:50Z", "author_association": "OWNER", "body": "For the moment, a workaround can be to `cat` an additional row onto the start of the file.\r\n\r\n    echo \"name,url,description\" | cat - missing_headings.csv | sqlite-utils insert blah.db table - --csv", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/228#issuecomment-778349672", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/228", "id": 778349672, "node_id": "MDEyOklzc3VlQ29tbWVudDc3ODM0OTY3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-02-12T18:00:43Z", "updated_at": "2021-02-12T18:00:43Z", "author_association": "OWNER", "body": "I could combine this with #131 to allow types to be specified in addition to column names.\r\n\r\nProbably need an option that means \"ignore the existing heading row and use this one instead\".", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 807437089, "label": "--no-headers option for CSV and TSV"}, "performed_via_github_app": null}