{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-864476167", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 864476167, "node_id": "MDEyOklzc3VlQ29tbWVudDg2NDQ3NjE2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-19T23:36:48Z", "updated_at": "2021-06-19T23:36:48Z", "author_association": "OWNER", "body": "Wrote this up on my blog here: https://simonwillison.net/2021/Jun/19/sqlite-utils-memory/ - with a video demo here: https://www.youtube.com/watch?v=OUjd0rkc678", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-864101267", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 864101267, "node_id": "MDEyOklzc3VlQ29tbWVudDg2NDEwMTI2Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-18T15:01:41Z", "updated_at": "2021-06-18T15:01:41Z", "author_association": "OWNER", "body": "I'll split the remaining work out into separate issues.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862491016", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862491016, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjQ5MTAxNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T15:46:13Z", "updated_at": "2021-06-16T15:46:13Z", "author_association": "OWNER", "body": "Columns from data imported from CSV in this way is currently treated as `TEXT`, which means numeric sorts and suchlike won't work as people might expect. It would be good to do automatic type detection here, see #179.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862485408", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862485408, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjQ4NTQwOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T15:38:58Z", "updated_at": "2021-06-16T15:39:28Z", "author_association": "OWNER", "body": "Also `sqlite-utils memory` reflects the existing `sqlite-utils :memory:` mechanism, which is a point in its favour.\r\n\r\nAnd it helps emphasize that the file you are querying will be loaded into memory, so probably don't try this against a 1GB CSV file.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862484557", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862484557, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjQ4NDU1Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T15:37:51Z", "updated_at": "2021-06-16T15:38:34Z", "author_association": "OWNER", "body": "I wonder if there's a better name for this than `sqlite-utils memory`?\r\n\r\n- `sqlite-utils memory hello.csv \"select * from hello\"`\r\n- `sqlite-utils mem hello.csv \"select * from hello\"`\r\n- `sqlite-utils temp hello.csv \"select * from hello\"`\r\n- `sqlite-utils adhoc hello.csv \"select * from hello\"`\r\n- `sqlite-utils scratch hello.csv \"select * from hello\"`\r\n\r\nI think `memory` is best. I don't like the others, except for `scratch` which is OK.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862479704", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862479704, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjQ3OTcwNA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T15:31:31Z", "updated_at": "2021-06-16T15:31:31Z", "author_association": "OWNER", "body": "Plus, could I make this change to `sqlite-utils query` without breaking backwards compatibility? Adding a new `sqlite-utils memory` command is completely safe from that perspective.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862478881", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862478881, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjQ3ODg4MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T15:30:24Z", "updated_at": "2021-06-16T15:30:24Z", "author_association": "OWNER", "body": "But... `sqlite-utils my.csv \"select * from my\"` is a much more compelling initial experience than `sqlite-utils memory my.csv \"select * from my\"`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862475685", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862475685, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjQ3NTY4NQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T15:26:19Z", "updated_at": "2021-06-16T15:29:38Z", "author_association": "OWNER", "body": "Here's a radical idea: what if I combined `sqlite-utils memory` into `sqlite-utils query`?\r\n\r\nThe trick here would be to detect if the arguments passed on the command-line refer to SQLite databases or if they refer to CSV/JSON data that should be imported into temporary tables.\r\n\r\nDetecting a SQLite database file is actually really easy - they all start with the same binary string:\r\n```pycon\r\n>>> open(\"my.db\", \"rb\").read(100)\r\nb'SQLite format 3\\x00...\r\n```\r\n(Need to carefully check that a CSV file with`SQLite format 3` as the first column name doesn't accidentally get interpreted as a SQLite DB though).\r\n\r\nSo then what would the semantics of `sqlite-utils query` (which is also the default command) be?\r\n\r\n- `sqlite-utils mydb.db \"select * from x\"`\r\n- `sqlite-utils my.csv \"select * from my\"`\r\n- `sqlite-utils mydb.db my.csv \"select * from mydb.x join my on ...\"` - this is where it gets weird. We can't import the CSV data directly into `mpdb.db` - it's suppose to go into the in-memory database - so now we need to start using database aliases like `mydb.x` because we passed at least one other file?\r\n\r\nThe complexity here is definitely in the handling of a combination of SQLite database files and CSV filenames. Also, `sqlite-utils query` doesn't accept multiple filenames at the moment, so that will change.\r\n\r\nI'm not 100% sold on this as being better than having a separate `sqlite-utils memory` command, as seen in #273.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862040971", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862040971, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjA0MDk3MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T05:02:56Z", "updated_at": "2021-06-16T05:02:56Z", "author_association": "OWNER", "body": "Moving this to a PR.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862040906", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862040906, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjA0MDkwNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T05:02:47Z", "updated_at": "2021-06-16T05:02:47Z", "author_association": "OWNER", "body": "Got a prototype working!\r\n```\r\n % curl -s 'https://fivethirtyeight.datasettes.com/polls/president_approval_polls.csv?_size=max&_stream=1' | sqlite-utils memory - 'select * from t limit 5' --nl \r\n{\"rowid\": \"1\", \"question_id\": \"139304\", \"poll_id\": \"74225\", \"state\": \"\", \"politician_id\": \"11\", \"politician\": \"Donald Trump\", \"pollster_id\": \"568\", \"pollster\": \"YouGov\", \"sponsor_ids\": \"352\", \"sponsors\": \"Economist\", \"display_name\": \"YouGov\", \"pollster_rating_id\": \"391\", \"pollster_rating_name\": \"YouGov\", \"fte_grade\": \"B\", \"sample_size\": \"1500\", \"population\": \"a\", \"population_full\": \"a\", \"methodology\": \"Online\", \"start_date\": \"1/16/21\", \"end_date\": \"1/19/21\", \"sponsor_candidate\": \"\", \"tracking\": \"\", \"created_at\": \"1/20/21 10:18\", \"notes\": \"\", \"url\": \"https://docs.cdn.yougov.com/y9zsit5bzd/weeklytrackingreport.pdf\", \"source\": \"538\", \"yes\": \"42.0\", \"no\": \"53.0\"}\r\n{\"rowid\": \"2\", \"question_id\": \"139305\", \"poll_id\": \"74225\", \"state\": \"\", \"politician_id\": \"11\", \"politician\": \"Donald Trump\", \"pollster_id\": \"568\", \"pollster\": \"YouGov\", \"sponsor_ids\": \"352\", \"sponsors\": \"Economist\", \"display_name\": \"YouGov\", \"pollster_rating_id\": \"391\", \"pollster_rating_name\": \"YouGov\", \"fte_grade\": \"B\", \"sample_size\": \"1155\", \"population\": \"rv\", \"population_full\": \"rv\", \"methodology\": \"Online\", \"start_date\": \"1/16/21\", \"end_date\": \"1/19/21\", \"sponsor_candidate\": \"\", \"tracking\": \"\", \"created_at\": \"1/20/21 10:18\", \"notes\": \"\", \"url\": \"https://docs.cdn.yougov.com/y9zsit5bzd/weeklytrackingreport.pdf\", \"source\": \"538\", \"yes\": \"44.0\", \"no\": \"55.0\"}\r\n{\"rowid\": \"3\", \"question_id\": \"139306\", \"poll_id\": \"74226\", \"state\": \"\", \"politician_id\": \"11\", \"politician\": \"Donald Trump\", \"pollster_id\": \"23\", \"pollster\": \"American Research Group\", \"sponsor_ids\": \"\", \"sponsors\": \"\", \"display_name\": \"American Research Group\", \"pollster_rating_id\": \"9\", \"pollster_rating_name\": \"American Research Group\", \"fte_grade\": \"B\", \"sample_size\": \"1100\", \"population\": \"a\", \"population_full\": \"a\", \"methodology\": \"Live Phone\", \"start_date\": \"1/16/21\", \"end_date\": \"1/19/21\", \"sponsor_candidate\": \"\", \"tracking\": \"\", \"created_at\": \"1/20/21 10:18\", \"notes\": \"\", \"url\": \"https://americanresearchgroup.com/economy/\", \"source\": \"538\", \"yes\": \"30.0\", \"no\": \"66.0\"}\r\n{\"rowid\": \"4\", \"question_id\": \"139307\", \"poll_id\": \"74226\", \"state\": \"\", \"politician_id\": \"11\", \"politician\": \"Donald Trump\", \"pollster_id\": \"23\", \"pollster\": \"American Research Group\", \"sponsor_ids\": \"\", \"sponsors\": \"\", \"display_name\": \"American Research Group\", \"pollster_rating_id\": \"9\", \"pollster_rating_name\": \"American Research Group\", \"fte_grade\": \"B\", \"sample_size\": \"990\", \"population\": \"rv\", \"population_full\": \"rv\", \"methodology\": \"Live Phone\", \"start_date\": \"1/16/21\", \"end_date\": \"1/19/21\", \"sponsor_candidate\": \"\", \"tracking\": \"\", \"created_at\": \"1/20/21 10:18\", \"notes\": \"\", \"url\": \"https://americanresearchgroup.com/economy/\", \"source\": \"538\", \"yes\": \"29.0\", \"no\": \"67.0\"}\r\n{\"rowid\": \"5\", \"question_id\": \"139298\", \"poll_id\": \"74224\", \"state\": \"\", \"politician_id\": \"11\", \"politician\": \"Donald Trump\", \"pollster_id\": \"1528\", \"pollster\": \"AtlasIntel\", \"sponsor_ids\": \"\", \"sponsors\": \"\", \"display_name\": \"AtlasIntel\", \"pollster_rating_id\": \"546\", \"pollster_rating_name\": \"AtlasIntel\", \"fte_grade\": \"B/C\", \"sample_size\": \"5188\", \"population\": \"a\", \"population_full\": \"a\", \"methodology\": \"Online\", \"start_date\": \"1/15/21\", \"end_date\": \"1/19/21\", \"sponsor_candidate\": \"\", \"tracking\": \"\", \"created_at\": \"1/19/21 21:52\", \"notes\": \"\", \"url\": \"https://projects.fivethirtyeight.com/polls/20210119_US_Atlas2.pdf\", \"source\": \"538\", \"yes\": \"44.6\", \"no\": \"53.9\"}\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-862018937", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 862018937, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MjAxODkzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T03:59:28Z", "updated_at": "2021-06-16T04:00:05Z", "author_association": "OWNER", "body": "Mainly for debugging purposes it would be useful to be able to save the created in-memory database back to a file again later. This could be done with:\r\n\r\n    sqlite-utils memory blah.csv --save saved.db\r\n\r\nCan use `.iterdump()` to implement this: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.iterdump\r\n\r\nMaybe instead (or as-well-as) offer `--dump` which dumps out the SQL from that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861989987", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861989987, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTk4OTk4Nw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T02:34:21Z", "updated_at": "2021-06-16T02:34:21Z", "author_association": "OWNER", "body": "The documentation already covers this\r\n```\r\n$ sqlite-utils :memory: \"select sqlite_version()\"\r\n[{\"sqlite_version()\": \"3.29.0\"}]\r\n```\r\nhttps://sqlite-utils.datasette.io/en/latest/cli.html#running-queries-and-returning-json\r\n\r\n`sqlite-utils memory \"select sqlite_version()\"` is a little bit more intuitive than that.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861987651", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861987651, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTk4NzY1MQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T02:27:20Z", "updated_at": "2021-06-16T02:27:20Z", "author_association": "OWNER", "body": "Solution: `sqlite-utils memory -` attempts to detect the input based on if it starts with a `{` or `[` (likely JSON) or if it doesn't use the `csv.Sniffer()` mechanism. Or you can use `sqlite-utils memory -:csv` to specifically indicate the type of input.", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861985944", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861985944, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTk4NTk0NA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T02:22:52Z", "updated_at": "2021-06-16T02:22:52Z", "author_association": "OWNER", "body": "Another option: allow an optional `:suffix` specifying the type of the file. If this is missing we detect based on the filename.\r\n\r\n    sqlite-utils memory somefile:csv \"select * from somefile\"\r\n\r\nOne catch: how to treat `-` for standard input?\r\n\r\n    cat blah.csv | sqlite-utils memory - \"select * from stdin\"\r\n\r\nThat's fine for CSV, but what about TSV or JSON or nl-JSON? Maybe this:\r\n\r\n    cat blah.csv | sqlite-utils memory -:json \"select * from stdin\"\r\n\r\nBit weird though. The alternative would be to support this:\r\n\r\n    cat blah.csv | sqlite-utils memory --load-csv -\r\n\r\nBut that's verbose compared to the version without the long `--load-x` option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861984707", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861984707, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTk4NDcwNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-16T02:19:48Z", "updated_at": "2021-06-16T02:19:48Z", "author_association": "OWNER", "body": "This is going to need to be a separate command, for relatively non-obvious reasons.\r\n\r\n    sqlite-utils blah.db \"select * from x\"\r\n\r\nIs equivalent to this, because `query` is the default sub-command:\r\n\r\n    sqlite-utils query blah.db \"select * from x\"\r\n\r\nBut... this means that making the filename optional doesn't actually work - because then this is ambiguous:\r\n\r\n    sqlite-utils --load-csv blah.csv \"select * from blah\"\r\n\r\nSo instead, I'm going to add a new sub-command. I'm currently thinking `memory` to reflect that this command operates on an in-memory database:\r\n\r\n    sqlite-utils memory --load-csv blah.csv \"select * from blah\"\r\n\r\nI still think I need to use `--load-csv` rather than `--csv` because one interesting use-case for this is loading in CSV and converting it to JSON, or vice-versa.\r\n\r\nAnother option: allow multiple arguments which are filenames, and use the extension (or sniff the content) to decide what to do with them:\r\n\r\n    sqlite-utils memory blah.csv foo.csv \"select * from foo join blah on ...\"\r\n\r\nThis would require the last positional argument to always be a SQL query, and would treat all other positional arguments as files that should be imported into memory.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861944202", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861944202, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTk0NDIwMg==", "user": {"value": 25778, "label": "eyeseast"}, "created_at": "2021-06-16T01:41:03Z", "updated_at": "2021-06-16T01:41:03Z", "author_association": "CONTRIBUTOR", "body": "So, I do things like this a lot, too. I like the idea of piping in from stdin. Something like this would be nice to do in a makefile:\r\n\r\n```sh\r\ncat file.csv | sqlite-utils --csv --table data - 'SELECT * FROM data WHERE col=\"whatever\"' > filtered.csv\r\n```\r\n\r\nIf you assumed that you're always piping out the same format you're piping in, the option names don't have to change. Depends how much you want to change formats.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861891835", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861891835, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTg5MTgzNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-15T23:09:31Z", "updated_at": "2021-06-15T23:09:31Z", "author_association": "OWNER", "body": "`--load-csv` and `--load-json` and `--load-nl` and `--load-tsv` are unambiguous.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861891693", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861891693, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTg5MTY5Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-15T23:09:08Z", "updated_at": "2021-06-15T23:09:08Z", "author_association": "OWNER", "body": "Problem: `--csv` and `--json` and `--nl` are already options for `sqlite-utils query` - need new non-conflicting names.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861891272", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861891272, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTg5MTI3Mg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-15T23:08:02Z", "updated_at": "2021-06-15T23:08:02Z", "author_association": "OWNER", "body": "`--csv -` should work though, for reading from stdin. The table can be called `stdin`.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861891110", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861891110, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTg5MTExMA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-15T23:07:38Z", "updated_at": "2021-06-15T23:07:38Z", "author_association": "OWNER", "body": "`--csvt` seems unnecessary to me: if people want to load different CSV files with the same filename (but in different directories) they will get an error unless they rename the files first.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861890689", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861890689, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTg5MDY4OQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-15T23:06:37Z", "updated_at": "2021-06-15T23:06:37Z", "author_association": "OWNER", "body": "How about `--json` and `--nl` and `--tsv` too? Imitating the format options for `sqlite-utils insert`.\r\n\r\nAnd what happens if you provide a filename too? I'm tempted to say that the `--csv` stuff still gets loaded into an in-memory database but it's given a name and can then be joined against using SQLite `memory.blah` syntax.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/272#issuecomment-861889437", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/272", "id": 861889437, "node_id": "MDEyOklzc3VlQ29tbWVudDg2MTg4OTQzNw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-06-15T23:03:26Z", "updated_at": "2021-06-15T23:03:26Z", "author_association": "OWNER", "body": "Maybe also support `--csvt` as an alternative option which takes two arguments: the CSV path and the name of the table that should be created from it (rather than auto-detecting from the filename).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 921878733, "label": "Idea: import CSV to memory, run SQL, export in a single command"}, "performed_via_github_app": null}