Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added the Send trait bound to all stream types #14

Merged
merged 1 commit into from
Jan 5, 2024
Merged

Added the Send trait bound to all stream types #14

merged 1 commit into from
Jan 5, 2024

Conversation

varonroy
Copy link
Contributor

@varonroy varonroy commented Jan 5, 2024

The goal of this PR is to allow streams to work from tokio::spawn.

Consider the following example:

async fn stream_from_spawn() {
    tokio::spawn(async move {
        let ollama = Arc::new(Ollama::default());

        let mut stream = Ollama::default()
            .send_chat_messages_stream(ChatMessageRequest::new("mistral".to_string(), vec![]))
            .await
            .unwrap();

        let _next = stream.next().await;
    });
}

Since spawn requires its futures to be Send, the above code will not compile and produce the following error message:

error: future cannot be sent between threads safely
   --> src/main.rs:7:5
    |
7   |     tokio::spawn(async move {
    |     ^^^^^^^^^^^^ future created by async block is not `Send`
    |
    = help: the trait `std::marker::Send` is not implemented for `dyn tokio_stream::Stream<Item = Result<ChatMessageResponse, ()>>`
note: future is not `Send` as this value is used across an await

The simplest solution is to add the Send trait bound to all stream types.

@pepperoni21 pepperoni21 merged commit d7ffa70 into pepperoni21:master Jan 5, 2024
2 checks passed
@pepperoni21
Copy link
Owner

pepperoni21 commented Jan 5, 2024

Thanks for your contribution!

@functorism
Copy link
Contributor

functorism commented Mar 17, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants