{"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991381679", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991381679, "node_id": "IC_kwDOCGYnMM47F0iv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T23:58:43Z", "updated_at": "2021-12-10T23:59:35Z", "author_association": "OWNER", "body": "I think the fix for this is to change the rules about what code is accepted in both the `-` mode and the literal code string mode: you can pass in a Python expression, OR a fragment that gets turned into a function, OR code that implements its own `def convert(value)` function. So this would work too:\r\n```sh\r\nsqlite-utils convert my.db mytable col1 '\r\ndef convert(value):\r\n    return value.upper()\r\n'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991381281", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991381281, "node_id": "IC_kwDOCGYnMM47F0ch", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T23:57:26Z", "updated_at": "2021-12-10T23:57:26Z", "author_association": "OWNER", "body": "My first attempt at building this looked a little bit strange, because you would end up having a file like this `convert.py`:\r\n\r\n```\r\nvalue = value.upper()\r\nreturn value\r\n```\r\nWhich gets used like this:\r\n\r\n    cat convert.py | sqlite-utils convert my.db mytable col1 -\r\n\r\nBut... that `convert.py` code isn't actually valid Python - it's a weird thing where you have a partial snippet of Python code that gets wrapped in a function automatically.\r\n\r\nIt would be better if you could write `convert.py` as a valid Python file with a function in it, something like this:\r\n```python\r\ndef convert(value):\r\n    value = value.upper()\r\n    return value\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991378346", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991378346, "node_id": "IC_kwDOCGYnMM47Fzuq", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T23:48:28Z", "updated_at": "2021-12-10T23:48:28Z", "author_association": "OWNER", "body": "One option: allow `CODE` to be a special value of `-` which means \"read from standard input\". It's a tiny bit of a hack but I think it would work here.\r\n\r\nIf you wanted to replace a column entirely with hyphens you would still be able to do this:\r\n\r\n    sqlite-utils convert my.db mytable col1 '\"-\"'", "reactions": "{\"total_count\": 1, \"+1\": 1, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991377288", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991377288, "node_id": "IC_kwDOCGYnMM47FzeI", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T23:45:53Z", "updated_at": "2021-12-10T23:45:53Z", "author_association": "OWNER", "body": "One challenge here: the current signature looks like this:\r\n```\r\n% sqlite-utils convert --help\r\nUsage: sqlite-utils convert [OPTIONS] DB_PATH TABLE COLUMNS... CODE\r\n```\r\n`CODE` is a positional argument which comes last - and since `COLUMNS` can be one or more items, making `CODE` optional isn't easy.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/353#issuecomment-991376639", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/353", "id": 991376639, "node_id": "IC_kwDOCGYnMM47FzT_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T23:43:45Z", "updated_at": "2021-12-10T23:43:45Z", "author_association": "OWNER", "body": "There's a very non-obvious workaround for this at the moment. You can save your code in e.g. a file called` transform.py` - my test one looks like this:\r\n```python\r\ndef upper(value):\r\n   return value.upper()\r\n```\r\nThen you can run the following to import and use that function:\r\n\r\n`PYTHONPATH=. sqlite-utils convert fixtures.db roadside_attractions name 'transform.upper(value)' --import transform`\r\n\r\nThat `PYTHONPATH=. bit is necessary because otherwise the script won't look in the current directory for that `transform.py` module.\r\n\r\nNow that I've written this down, it's obviously bad! I think your suggestion here is a good idea.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077102934, "label": "Allow passing a file of code to \"sqlite-utils convert\""}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/354#issuecomment-991309759", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354", "id": 991309759, "node_id": "IC_kwDOCGYnMM47Fi-_", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T21:33:18Z", "updated_at": "2021-12-10T21:33:18Z", "author_association": "OWNER", "body": "https://www.sqlite.org/fts5.html#the_rebuild_command says:\r\n\r\n> This command first deletes the entire full-text index, then rebuilds it based on the contents of the table or [content table](https://www.sqlite.org/fts5.html#external_content_tables). It is not available with [contentless tables](https://www.sqlite.org/fts5.html#contentless_tables).\r\n> \r\n> `INSERT INTO ft(ft) VALUES('rebuild');`", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077243232, "label": "Test failure in test_rebuild_fts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/354#issuecomment-991309002", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354", "id": 991309002, "node_id": "IC_kwDOCGYnMM47FizK", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T21:32:14Z", "updated_at": "2021-12-10T21:32:14Z", "author_association": "OWNER", "body": "Here's what the method does: https://github.com/simonw/sqlite-utils/blob/e328db8eba1fbf29a69eda95dfec861954f9e771/sqlite_utils/db.py#L1941-L1952\r\n\r\nMaybe I don't need a test that deliberately corrupts the database here? Not sure how to test that `rebuild` has been called though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077243232, "label": "Test failure in test_rebuild_fts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/354#issuecomment-991307422", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354", "id": 991307422, "node_id": "IC_kwDOCGYnMM47Fiae", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T21:29:34Z", "updated_at": "2021-12-10T21:29:34Z", "author_association": "OWNER", "body": "Here's the test in question. The way it works is a bit weird (deleting everything in the `_fts_data` table in order to force errors that can be fixed with `.rebuild_fts()`): https://github.com/simonw/sqlite-utils/blob/8ae77a6961fed94ef2c9cc81fcfc7c81d222d9a2/tests/test_fts.py#L257-L285", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077243232, "label": "Test failure in test_rebuild_fts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/354#issuecomment-991306712", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/354", "id": 991306712, "node_id": "IC_kwDOCGYnMM47FiPY", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T21:28:27Z", "updated_at": "2021-12-10T21:28:27Z", "author_association": "OWNER", "body": "Failures started with this commit, which only touches documentation so is completely unrelated: https://github.com/simonw/sqlite-utils/commit/e328db8eba1fbf29a69eda95dfec861954f9e771", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1077243232, "label": "Test failure in test_rebuild_fts"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/1518#issuecomment-991285527", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1518", "id": 991285527, "node_id": "IC_kwDOBm6k_c47FdEX", "user": {"value": 9599, "label": "simonw"}, "created_at": "2021-12-10T20:52:00Z", "updated_at": "2021-12-10T20:52:00Z", "author_association": "OWNER", "body": "If I break this up into `@inject` methods, what methods could I have and what would they do?\r\n\r\n- `resolve_path`: Use request path to resolve the database and table. Could handle hash URLs too (if I don't manage to extract those to a plugin) - would be nice if this could raise a redirect, but I think that will instead have to be one of the things it returns\r\n- `build_sql`: Builds the SQL query based on the querystring (and some DB introspection)\r\n- `execute_count`: Execute the `count(*)`\r\n- `execute_rows`: Execute the `limit 101` to fetch the rows\r\n- `execute_facets`: Execute all requested facets (could this do its own `asyncio.gather()` to run facets in parallel?)\r\n- `suggest_facets`: Execute facet suggestions\r\n\r\nAre there any plugin hooks that would make sense to execute in parallel? Actually there might be: I don't think `extra_template_vars`, `extra_css_urls`, `extra_js_urls`, `extra_body_script` depend on each other so it might be possible to execute them in a parallel chunk (at least any of them that return awaitables).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1058072543, "label": "Complete refactor of TableView and table.html template"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/pull/1548#issuecomment-990967417", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/1548", "id": 990967417, "node_id": "IC_kwDOBm6k_c47EPZ5", "user": {"value": 22429695, "label": "codecov[bot]"}, "created_at": "2021-12-10T13:19:00Z", "updated_at": "2021-12-10T13:19:00Z", "author_association": "NONE", "body": "# [Codecov](https://codecov.io/gh/simonw/datasette/pull/1548?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) Report\n> Merging [#1548](https://codecov.io/gh/simonw/datasette/pull/1548?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (68383f5) into [main](https://codecov.io/gh/simonw/datasette/commit/737115ea14cd51ffb55dea886e6a684c148db2c9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison) (737115e) will **not change** coverage.\n> The diff coverage is `n/a`.\n\n[![Impacted file tree graph](https://codecov.io/gh/simonw/datasette/pull/1548/graphs/tree.svg?width=650&height=150&src=pr&token=eSahVY7kw1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)](https://codecov.io/gh/simonw/datasette/pull/1548?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n\n```diff\n@@           Coverage Diff           @@\n##             main    #1548   +/-   ##\n=======================================\n  Coverage   91.84%   91.84%           \n=======================================\n  Files          34       34           \n  Lines        4438     4438           \n=======================================\n  Hits         4076     4076           \n  Misses        362      362           \n```\n\n\n\n------\n\n[Continue to review full report at Codecov](https://codecov.io/gh/simonw/datasette/pull/1548?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison)\n> `\u0394 = absolute <relative> (impact)`, `\u00f8 = not affected`, `? = missing data`\n> Powered by [Codecov](https://codecov.io/gh/simonw/datasette/pull/1548?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Last update [737115e...68383f5](https://codecov.io/gh/simonw/datasette/pull/1548?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Simon+Willison).\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1076834768, "label": "Update pytest-xdist requirement from <2.5,>=2.2.1 to >=2.2.1,<2.6"}, "performed_via_github_app": null}