{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239772256", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239772256, "node_id": "IC_kwDOCGYnMM5J5Wxg", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T19:07:51Z", "updated_at": "2022-09-07T19:09:52Z", "author_association": "OWNER", "body": "Or... I could automatically register multiple copies of the function of different arities!\r\n\r\nIf I'm going to do something like that though I need to think carefully about how functions that have keyword-only arguments should work: https://peps.python.org/pep-3102/\r\n\r\n```python\r\ndef compare(a, b, *ignore, key=None):\r\n    ...\r\n```\r\nI should think about how these work with `db.register_function()` anyway, since SQL functions cannot support keyword arguments.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239766987", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239766987, "node_id": "IC_kwDOCGYnMM5J5VfL", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T19:01:49Z", "updated_at": "2022-09-07T19:01:49Z", "author_association": "OWNER", "body": "OK with this:\r\n```diff\r\ndiff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py\r\nindex c51b101..93d82a9 100644\r\n--- a/sqlite_utils/cli.py\r\n+++ b/sqlite_utils/cli.py\r\n@@ -30,6 +30,7 @@ from .utils import (\r\n     Format,\r\n     TypeTracker,\r\n )\r\n+from . import recipes\r\n \r\n \r\n CONTEXT_SETTINGS = dict(help_option_names=[\"-h\", \"--help\"])\r\n@@ -3029,7 +3030,7 @@ def _load_extensions(db, load_extension):\r\n def _register_functions(db, functions):\r\n     # Register any Python functions as SQL functions:\r\n     sqlite3.enable_callback_tracebacks(True)\r\n-    globals = {}\r\n+    globals = {\"r\": recipes, \"recipes\": recipes}\r\n     try:\r\n         exec(functions, globals)\r\n     except SyntaxError as ex:\r\n@@ -3037,4 +3038,4 @@ def _register_functions(db, functions):\r\n     # Register all callables in the locals dict:\r\n     for name, value in globals.items():\r\n         if callable(value) and not name.startswith(\"_\"):\r\n-            db.register_function(value, name=name)\r\n+            db.register_function(value, name=name, ignore_defaults=True)\r\ndiff --git a/sqlite_utils/db.py b/sqlite_utils/db.py\r\nindex 27c46b0..1407d23 100644\r\n--- a/sqlite_utils/db.py\r\n+++ b/sqlite_utils/db.py\r\n@@ -370,6 +370,7 @@ class Database:\r\n         self,\r\n         fn: Callable = None,\r\n         deterministic: bool = False,\r\n+        ignore_defaults: bool = False,\r\n         replace: bool = False,\r\n         name: Optional[str] = None,\r\n     ):\r\n@@ -397,7 +398,10 @@ class Database:\r\n \r\n         def register(fn):\r\n             fn_name = name or fn.__name__\r\n-            arity = len(inspect.signature(fn).parameters)\r\n+            params = inspect.signature(fn).parameters\r\n+            if ignore_defaults:\r\n+                params = [p for p in params if params[p].default is inspect._empty]\r\n+            arity = len(params)\r\n             if not replace and (fn_name, arity) in self._registered_functions:\r\n                 return fn\r\n             kwargs = {}\r\n```\r\nI can now do this:\r\n```\r\n% sqlite-utils memory --functions 'parsedate = r.parsedate' 'select parsedate(\"1st jan\")'\r\n[{\"parsedate(\\\"1st jan\\\")\": \"2022-01-01\"}]\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239763997", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239763997, "node_id": "IC_kwDOCGYnMM5J5Uwd", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T18:57:59Z", "updated_at": "2022-09-07T18:58:10Z", "author_association": "OWNER", "body": "Here's how to detect defaults in the function signature:\r\n```pycon\r\n>>> import inspect\r\n>>> def foo(a, b, c=1, d=2):\r\n...      pass\r\n... \r\n>>> inspect.signature(foo)\r\n<Signature (a, b, c=1, d=2)>\r\n>>> inspect.signature(foo).parameters\r\nmappingproxy(OrderedDict([('a', <Parameter \"a\">), ('b', <Parameter \"b\">), ('c', <Parameter \"c=1\">), ('d', <Parameter \"d=2\">)]))\r\n>>> inspect.signature(foo).parameters['c']\r\n<Parameter \"c=1\">\r\n>>> dir(inspect.signature(foo).parameters['c'])\r\n['KEYWORD_ONLY', 'POSITIONAL_ONLY', 'POSITIONAL_OR_KEYWORD', 'VAR_KEYWORD', 'VAR_POSITIONAL', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', '_annotation', '_default', '_kind', '_name', 'annotation', 'default', 'empty', 'kind', 'name', 'replace']\r\n>>> inspect.signature(foo).parameters['c'].default\r\n1\r\n>>> inspect.signature(foo).parameters['a'].default\r\n<class 'inspect._empty'>\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239762561", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239762561, "node_id": "IC_kwDOCGYnMM5J5UaB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T18:56:13Z", "updated_at": "2022-09-07T18:56:13Z", "author_association": "OWNER", "body": "I could do this:\r\n```python\r\n # Register all callables in the locals dict: \r\n for name, value in globals.items(): \r\n     if callable(value) and not name.startswith(\"_\"): \r\n         db.register_function(value, name=name, ignore_params_with_defaults=True) \r\n```\r\nIntroducing a new `ignore_params_with_defaults` option.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239762031", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239762031, "node_id": "IC_kwDOCGYnMM5J5URv", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T18:55:30Z", "updated_at": "2022-09-07T18:55:30Z", "author_association": "OWNER", "body": "That would be a breaking change though - existing code that registers functions with default parameters should continue to work unchanged (unless I want to ship `sqlite-utils` 4.0).", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239761280", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239761280, "node_id": "IC_kwDOCGYnMM5J5UGA", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T18:54:51Z", "updated_at": "2022-09-07T18:54:51Z", "author_association": "OWNER", "body": "I could teach this code here to only register the function using arguments that don't have default parameters:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/d9b9e075f07a20f1137cd2e34ed5d3f1a3db4ad8/sqlite_utils/cli.py#L3037-L3040\r\n\r\nOr even this code here:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/d9b9e075f07a20f1137cd2e34ed5d3f1a3db4ad8/sqlite_utils/db.py#L398-L418\r\n", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239760001", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239760001, "node_id": "IC_kwDOCGYnMM5J5TyB", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T18:53:17Z", "updated_at": "2022-09-07T18:53:17Z", "author_association": "OWNER", "body": "So you would need to do this instead:\r\n```\r\nsqlite-utils memory 'select parsedate(\"1st jan\")' --functions '\r\ndef parsedate(s):\r\n    return r.parsedate(s)\r\n'\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239759022", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239759022, "node_id": "IC_kwDOCGYnMM5J5Tiu", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T18:52:08Z", "updated_at": "2022-09-07T18:52:08Z", "author_association": "OWNER", "body": "It's not quite that simple. I tried applying this patch:\r\n\r\n```diff\r\ndiff --git a/sqlite_utils/cli.py b/sqlite_utils/cli.py\r\nindex c51b101..33e4d90 100644\r\n--- a/sqlite_utils/cli.py\r\n+++ b/sqlite_utils/cli.py\r\n@@ -30,6 +30,7 @@ from .utils import (\r\n     Format,\r\n     TypeTracker,\r\n )\r\n+from . import recipes\r\n \r\n \r\n CONTEXT_SETTINGS = dict(help_option_names=[\"-h\", \"--help\"])\r\n@@ -3029,7 +3030,7 @@ def _load_extensions(db, load_extension):\r\n def _register_functions(db, functions):\r\n     # Register any Python functions as SQL functions:\r\n     sqlite3.enable_callback_tracebacks(True)\r\n-    globals = {}\r\n+    globals = {\"r\": recipes, \"recipes\": recipes}\r\n     try:\r\n         exec(functions, globals)\r\n     except SyntaxError as ex:\r\n```\r\n\r\nThen got this:\r\n\r\n```\r\n% sqlite-utils memory --functions 'parsedate = r.parsedate' 'select parsedate(\"1st jan\")'\r\nError: wrong number of arguments to function parsedate()\r\n% sqlite-utils memory --functions 'parsedate = r.parsedate' 'select parsedate(\"1st jan\", 0, 0, 0)' \r\n[{\"parsedate(\\\"1st jan\\\", 0, 0, 0)\": \"2022-01-01\"}]\r\n```\r\nThe problem here is that the `parsedate` function signature looks like this:\r\n\r\nhttps://github.com/simonw/sqlite-utils/blob/d9b9e075f07a20f1137cd2e34ed5d3f1a3db4ad8/sqlite_utils/recipes.py#L8\r\n\r\nBut the code that register SQL functions introspects that signature, so creates a SQL function that requires four arguments.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239699276", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239699276, "node_id": "IC_kwDOCGYnMM5J5E9M", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T17:49:49Z", "updated_at": "2022-09-07T17:49:49Z", "author_association": "OWNER", "body": "This feature is a tiny bit weird though: the recipe functions are not exposed to SQL by default, they are instead designed to be used with `sqlite-utils convert`. I guess with `--functions` support you could do something like this:\r\n\r\n    sqlite-utils data.db \"update mytable set col1 = parsedate(col1)\" --functions \"parsedate = r.parsedate\"", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1239697643", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1239697643, "node_id": "IC_kwDOCGYnMM5J5Ejr", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-07T17:48:00Z", "updated_at": "2022-09-07T17:48:00Z", "author_association": "OWNER", "body": "Will also need to update documentation here: https://sqlite-utils.datasette.io/en/stable/cli.html#defining-custom-sql-functions", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/sqlite-utils/issues/484#issuecomment-1238607591", "issue_url": "https://api.github.com/repos/simonw/sqlite-utils/issues/484", "id": 1238607591, "node_id": "IC_kwDOCGYnMM5J06bn", "user": {"value": 9599, "label": "simonw"}, "created_at": "2022-09-06T20:16:39Z", "updated_at": "2022-09-06T20:16:39Z", "author_association": "OWNER", "body": "Here's the implementation for recipes at the moment: https://github.com/simonw/sqlite-utils/blob/5b969273f1244b1bcf3e4dc071cdf17dab35d5f8/sqlite_utils/utils.py#L434-L441\r\n\r\nAnd here's the `--functions` implementation that doesn't expose them: https://github.com/simonw/sqlite-utils/blob/5b969273f1244b1bcf3e4dc071cdf17dab35d5f8/sqlite_utils/cli.py#L3029-L3040", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 1363766973, "label": "Expose convert recipes to `sqlite-utils --functions`"}, "performed_via_github_app": null}