issue_comments

Documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#returning-all-rows-in-a-table

Documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#returning-all-rows-in-a-table

sqlite-utils search has --limit already: https://sqlite-utils.datasette.io/en/latest/cli-reference.html#search

--limit INTEGER Number of rows to return - defaults to everything

Twitter thread about this: https://twitter.com/simonw/status/1481020195074293761

The new page of documentation: https://sqlite-utils.datasette.io/en/latest/cli-reference.html

Here's the cog code I used: https://github.com/simonw/sqlite-utils/blob/1d44b0cc2784c94aed1bcf350225cd86ee1aa7e5/docs/cli-reference.rst#L11-L76

I'll call the new page "CLI reference", for consistency with the API reference page here: https://sqlite-utils.datasette.io/en/stable/reference.html

CLI and Python library improvements to help run ANALYZE after creating indexes or inserting rows, to gain better performance from the SQLite query planner when it runs against indexes.

Three new CLI commands: create-database, analyze and bulk.

• New sqlite-utils create-database command for creating new empty database files. (#348)
• New Python methods for running ANALYZE against a database, table or index: db.analyze() and table.analyze(), see Optimizing index usage with ANALYZE. (#366)
• New sqlite-utils analyze command for running ANALYZE using the CLI. (#379)
• The create-index, insert and update commands now have a new --analyze option for running ANALYZE after the command has completed. (#379)
• New sqlite-utils bulk command which can import records in the same way as sqlite-utils insert (from JSON, CSV or TSV) and use them to bulk execute a parametrized SQL query. (#375)
• The CLI tool can now also be run using python -m sqlite_utils. (#368)
• Using --fmt now implies --table, so you don't need to pass both options. (#374)
• The --convert function applied to rows can now modify the row in place. (#371)
• The insert-files command supports two new columns: stem and suffix. (#372)
• The --nl import option now ignores blank lines in the input. (#376)
• Fixed bug where streaming input to the insert command with --batch-size 1 would appear to only commit after several rows had been ingested, due to unnecessary input buffering. (#364)

Documentation: https://sqlite-utils.datasette.io/en/latest/cli.html#executing-sql-in-bulk

I tested this like so: ``` % wget 'https://raw.githubusercontent.com/wri/global-power-plant-database/master/output_database/global_power_plant_database.csv' % sqlite-utils create-database test.db % sqlite-utils create-table test.db power_plants url text owner text % sqlite-utils schema test.db CREATE TABLE [power_plants] ( [url] TEXT, [owner] TEXT ); % sqlite-utils bulk test.db 'insert into power_plants (url, owner) values (:url, :owner)' global_power_plant_database.csv --csv [------------------------------------] 0% [###################################-] 99% % sqlite-utils tables --counts test.db -t table count

power_plants 33643 ```

Documentation: https://github.com/simonw/sqlite-utils/blob/f4ea0d32c0543373eefaa9b9f3911eb07549eecb/docs/cli.rst#executing-sql-in-bulk

I decided to go with making this opt-in, mainly for consistency with the other places where I added this feature - see: - #379 - #366

You can now run the following:

sqlite-utils create-index mydb.db mytable mycolumn --analyze

And ANALYZE will be run on the index once it has been created.

The Python methods are all done now, next step is the CLI options. I'll do those in a separate issue.