You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For LLM providers that allow setting the PRNG seed for sampling determinism reasons, Kurt should provide a way to do this.
Current expectations of support across LLM providers:
OpenAI has this as a documented feature
VertexAI Gemini doesn't appear to have it, but it may be an undocumented feature
Anthropic Claude doesn't appear to have it.
Local LLMs should be able to easily support it.
For LLMs which do not support this feature, if the optional seed parameter is given by the user, Kurt should throw a KurtFeatureIncompatibleError (an error class which doesn't yet exist, but I've been thinking about for some time). We should also include a row for this in the #28 to highlight which LLM API providers have and don't have this feature.
The text was updated successfully, but these errors were encountered:
For LLM providers that allow setting the PRNG seed for sampling determinism reasons, Kurt should provide a way to do this.
Current expectations of support across LLM providers:
For LLMs which do not support this feature, if the optional
seed
parameter is given by the user, Kurt should throw aKurtFeatureIncompatibleError
(an error class which doesn't yet exist, but I've been thinking about for some time). We should also include a row for this in the #28 to highlight which LLM API providers have and don't have this feature.The text was updated successfully, but these errors were encountered: