issue_comments

Yes this was absolutely the intention! Thanks, I wonder how often I've made that mistake in other projects?

Tested it like this, against a freshly built .tar.gz package from my development environment: (w) w % mypy . hello.py:1: error: Skipping analyzing "sqlite_utils": found module but no type hints or library stubs hello.py:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports Found 1 error in 1 file (checked 1 source file) (w) w % pip install ~/Dropbox/Development/sqlite-utils/dist/sqlite-utils-3.17.1.tar.gz Processing /Users/simon/Dropbox/Development/sqlite-utils/dist/sqlite-utils-3.17.1.tar.gz ... Successfully installed sqlite-utils-3.17.1 (w) w % mypy . Success: no issues found in 1 source file I tested against the .whl too.

My hello.py script contained this: ```python import sqlite_utils from typing import cast

if name == "main": db = sqlite_utils.Database(memory=True) table = cast(sqlite_utils.db.Table, db["foo"]) table.insert({"id": 5}) print(list(db.query("select * from foo"))) That `cast()` is necessary because without it you get this error: (w) w % mypy . hello.py:7: error: Item "View" of "Union[Table, View]" has no attribute "insert" ```

Thanks - I didn't know this was needed!

This is a great idea.

OK, this should fix it.

create_index(..., find_unique_name=) is good. Default to false. index_foreign_keys can set it to true.

I'm leaning towards table.create_index(columns, ignore_existing_name=True) now.

Or resolve_existing_name - or skip_existing_name?

"ignore" sounds like it might not create the index if the name exists, but we want to still create the index but pick a new name.

I think I'll attempt to create the index and re-try if it fails with that error.

How to figure out if an index name is already in use? PRAGMA index_list(t) requires a table name. This does it:

sql SELECT name FROM sqlite_master WHERE type = 'index';

Looking at the method signature: https://github.com/simonw/sqlite-utils/blob/92aa5c9c5d26b0889c8c3d97c76a908d5f8af211/sqlite_utils/db.py#L1518-L1524

if_not_exists just adds a IF NOT EXISTS clause here: https://github.com/simonw/sqlite-utils/blob/92aa5c9c5d26b0889c8c3d97c76a908d5f8af211/sqlite_utils/db.py#L1549-L1561

SQLite will happily create multiple identical indexes on a table, using more disk space each time: ```pycon

import sqlite_utils db = sqlite_utils.Database("dupes.db") db["t"].insert_all({"id": i} for i in range(10000))

I'm tempted to not provide an opt-out option either: if you call table.create_index(...) without specifying an index name I think the tool should create the index for you, quietly picking an index name that works.

But... it feels wasteful to create an index that exactly duplicates an existing index. Would SQLite even let you do that or would it notice and NOT double the amount of disk space used for that index?

What would such an option be called? Some options:

• table.create_index([fk.column], force=True) - not obvious what force means here
• table.create_index([fk.column], ignore_existing_name=True) - not obvious what ignore means here
• table.create_index([fk.column], pick_unique_name=True) - bit verbose

If the user doesn't pass in an explicit name it seems like their intent is "just create me the index, I don't care what name you use" - so actually perhaps the default behaviour here should be to pick a new unique name if that name is already in use.

Then maybe there should be an option to disable that - some options there:

• table.create_index([fk.column], error_on_existing_index_name=True) - too verbose
• table.create_index([fk.column], force=False) - not clear what force means

The "index idx_generators_eia860_report_date already exists" error suggests that the problem here is actually one of an index name collision.

python table.create_index([fk.column]) This will derive a name for the index automatically from the name of the table and the name of the passed in columns: https://github.com/simonw/sqlite-utils/blob/92aa5c9c5d26b0889c8c3d97c76a908d5f8af211/sqlite_utils/db.py#L1536-L1539

So perhaps .create_index() should grow an extra option that creates the index even if the name already exists, by finding a new name.

This is strange - the code already checks that an index doesn't exist before attempting to create it: https://github.com/simonw/sqlite-utils/blob/92aa5c9c5d26b0889c8c3d97c76a908d5f8af211/sqlite_utils/db.py#L893-L902

Here's the build with that fix: https://readthedocs.org/projects/datasette/builds/15268498/

It passed and published the docs: https://docs.datasette.io/en/latest/changelog.html

Here's the new build using Python 3: https://readthedocs.org/projects/datasette/builds/15268482/

It's still broken. Here's one of many issue threads about it, this one has a workaround fix: https://github.com/readthedocs/readthedocs.org/issues/8616#issuecomment-952034858

For future readers, the solution for this problem is to pin docutils<0.18 in your requirements.txt file, and have a .readthedocs.yaml file with these contents:

``` version: 2

python: install: - requirements: docs/requirements.txt ```

Looks like ReadTheDocs builds started failing for latest a few weeks ago:

I will try adding a .readthedocs.yml file: https://docs.readthedocs.io/en/stable/config-file/v2.html#python-version

This might work:

``` version: 2

build: os: ubuntu-20.04 tools: python: "3.9"

sphinx: configuration: docs/conf.py ```

https://blog.readthedocs.com/default-python-3/ they started defaulting new projects to Python 3 back in Feb 2019 but clearly my project was created before then.

It looks like my builds there still run on Python 2!

git clone --no-single-branch --depth 50 https://github.com/simonw/datasette . git checkout --force de1e031713f47fbd51eb7239db3e7e6025fbf81a git clean -d -f -f python2.7 -mvirtualenv /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2 /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m pip install --upgrade --no-cache-dir pip setuptools /home/docs/checkouts/readthedocs.org/user_builds/datasette/envs/0.59.2/bin/python -m pip install --upgrade --no-cache-dir mock==1.0.1 pillow==5.4.1 alabaster>=0.7,<0.8,!=0.7.5 commonmark==0.8.1 recommonmark==0.5.0 sphinx<2 sphinx-rtd-theme<0.5 readthedocs-sphinx-ext<2.2 cat docs/conf.py

Error: ``` def test_table_html_filter_form_column_options( path, expected_column_options, app_client ): response = app_client.get(path) assert response.status == 200 form = Soup(response.body, "html.parser").find("form") column_options = [ o.attrs.get("value") or o.string for o in form.select("select[name=_filter_column] option") ]

  assert expected_column_options == column_options

E AssertionError: assert ['- column -'...wid', 'value'] == ['- column -', 'value'] E At index 1 diff: 'rowid' != 'value' E Left contains one more item: 'value' E Use -v to get the full diff `` This is becauserowid` isn't a table column but IS returned by the query used on that page.

My solution: start with the query columns, but then add any table columns that were not already returned by the query to the end of the filter_columns list.

I think the answer is to spot this case and link to ?_item_exact=x instead of ?_item=x - it looks like that's already recommended in the documentation here: https://docs.datasette.io/en/stable/json_api.html#column-filter-arguments

?column__exact=value or ?_column=value Returns rows where the specified column exactly matches the value.

So maybe the facet selection rendering logic needs to spot this and link correctly to it?