Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NA] fix gemini streaming #1106

Merged
merged 3 commits into from
Jan 22, 2025
Merged

[NA] fix gemini streaming #1106

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4b0a8c8
fix gemini streaming
idoberko2 Jan 22, 2025
949ba6f
added missing comment
idoberko2 Jan 22, 2025
058a43d
pr comments
idoberko2 Jan 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions apps/opik-backend/src/main/java/com/comet/opik/domain/llmproviders/LlmProviderGemini.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import lombok.RequiredArgsConstructor;

import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.function.Consumer;

@RequiredArgsConstructor
Expand All @@ -27,9 +28,9 @@ public ChatCompletionResponse generate(@NonNull ChatCompletionRequest request, @
public void generateStream(@NonNull ChatCompletionRequest request, @NonNull String workspaceId,
@NonNull Consumer<ChatCompletionResponse> handleMessage, @NonNull Runnable handleClose,
@NonNull Consumer<Throwable> handleError) {
llmProviderClientGenerator.newGeminiStreamingClient(apiKey, request)
CompletableFuture.runAsync(() -> llmProviderClientGenerator.newGeminiStreamingClient(apiKey, request)
idoberko2 marked this conversation as resolved.
Show resolved Hide resolved
.generate(request.messages().stream().map(LlmProviderGeminiMapper.INSTANCE::toChatMessage).toList(),
new ChunkedResponseHandler(handleMessage, handleClose, handleError, request.model()));
new ChunkedResponseHandler(handleMessage, handleClose, handleError, request.model())));
}

@Override
Expand Down
2 changes: 1 addition & 1 deletion ...ckend/src/test/java/com/comet/opik/api/resources/v1/priv/ChatCompletionsResourceTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@
/// relevant test will be skipped for that provider.
/// - **Openai**: runs against a demo server and doesn't require an API key
/// - **Anthropic**: set `ANTHROPIC_API_KEY` to your anthropic api key
/// - **Gemini**: set `GEMINI_AI_KEY` to your gemini api key
idoberko2 marked this conversation as resolved.
Show resolved Hide resolved
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
// Disabled because the tests require an API key to run and this seems to be failing in the CI pipeline
class ChatCompletionsResourceTest {
Expand Down Expand Up @@ -187,7 +188,6 @@ void createReturnsBadRequestWhenModelIsInvalid(String model) {

@ParameterizedTest
@MethodSource("testModelsProvider")
@Disabled
idoberko2 marked this conversation as resolved.
Show resolved Hide resolved
void createAndStreamResponse(String expectedModel, LlmProvider llmProvider, String llmProviderApiKey) {
assumeThat(llmProviderApiKey).isNotEmpty();

Expand Down
Loading