Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add optional timeout property to ConversationHandlerFunctionProps and use in ConversationHandlerFunction #2349

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

kreuzhofer
Copy link

Problem

ConversationHandlerFunction times out when doing long running tool requests. timeout parameter is added to cover those edge cases by the developer. Running tools can take longer than 60 seconds for some special cases like generating assets like images or videos by other LLMs as a response to a tool request. Therefore the timeout for the conversation handler function should be flexible to be changed by the developer if needed.

Issue number, if available:

Changes

Introduce optional timeout parameter (typeof Duration) to ConversationHandlerFunctionProps and use in ConversationHandlerFunction constructor to define optionally another timeout than the default 60s.

Corresponding docs PR, if applicable:

Validation

Change has been tested locally with long running dummy tool that delays answer to 2 minutes.

Checklist

  • If this PR includes a functional change to the runtime behavior of the code, I have added or updated automated test coverage for this change.
  • If this PR requires a change to the Project Architecture README, I have included that update in this PR.
  • If this PR requires a docs update, I have linked to that docs PR above. - YOU DECIDE if this needs additional docs
  • If this PR modifies E2E tests, makes changes to resource provisioning, or makes SDK calls, I have run the PR checks with the run-e2e label set.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@kreuzhofer kreuzhofer requested review from a team as code owners December 18, 2024 09:44
Copy link

changeset-bot bot commented Dec 18, 2024

⚠️ No Changeset found

Latest commit: 812783d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant