feat: Adding LiteLLM Router support to dspy.LM with RoutedLM

[zhaohan-dong]

No description provided.

[zhaohan-dong]

Should probably add a dummy for testing if it looks good.

[okhat]

Hey @zhaohan-dong ! Thanks. This is awesome. Most of the code appears to be nearly an exact duplicate of dspy.LM, so it makes me wonder if there's a cleaner way to avoid duplication?

[zhaohan-dong]

😅 If I'm allowed to modify the LM class, I'd say this part from Line 38 can be extracted to another function:

        if self.model_type == "chat":
            completion = cached_litellm_completion if cache else litellm_completion
        else:
            completion = cached_litellm_text_completion if cache else litellm_text_completion

Like completion = extracted_function()
So the RoutedLM class can override only this extracted function.

Separately I'm thinking if there's any benefit of moving the router completion methods to standalone functions. But then there might be issue with the function arguments for caching.

[zhaohan-dong]

@okhat I feel like letting the litellm inference functions conditionally use routers would actually make it cleaner. It is possible since the litellm.Router can be hashed for lru_cache.

Let me know what you think!