Agent v2 #2773

futursolo · 2022-07-03T05:38:43Z

Description

This pull request consists of the following changes:

Adapts the Worker v2 format.
Agents are now managed via context providers.
Adds Oneshot Agent (For each agent a single input and a single output is expected)
Adds Reactor Agent (For each agent, multiple inputs and multiple outputs are expected)
~~SSR support for memorised Tasks.~~ (deferred for a later pull request)

This pull request needs the following pull requests:

Questions to solve:

In Rust, a running asynchronous coroutine is also commonly named task. This may cause confusion to users. I do not have a better name for this, so I am open to suggestions to an alternative name than task.

Still missing:

tests
documentation

Checklist

I have reviewed my own code
I have added tests

…r a subscription.

# Conflicts: # packages/yew-agent/src/hooks.rs

ranile

We should have a struct components example too

examples/README.md

ranile · 2023-08-29T16:42:28Z

examples/web_worker_fib/src/lib.rs

-            move |e| link.send_message(Self::Message::WorkerMsg(e))
-        };
-        let worker = Worker::bridge(Rc::new(cb));
+            spawn_local(async move {


I would like there to be a way to avoid calling spawn_local directly. Can we abstract over it somehow? Perhaps using suspense here such that the component renders when the value is available from the worker?

A Suspense-capable hook should also be SSR-capable.
Currently, I haven't found a good way to provide the same guarantee of agents in SSR.
In particular, global static is different for each agents spawned and shared for all the forked agents, but that is not true for SSR even if they are spawned into a dedicated thread.
To provide the same level of isolation, we need to spawn child processes in SSR, which is more complicated.

In addition, Suspense should only be used to resolve query (idempotent) futures, not mutations.
We cannot guarantee all oneshot agents don't have side-effects and can be safely executed as many times as the the renderer wants to. So it is needed to provide a mutation-like hook where the output can be executed via a future.

ranile · 2023-08-29T16:51:00Z

examples/web_worker_prime/index.html

+    <title>Yew • Web Worker Prime</title>
+
+    <link data-trunk rel="rust" href="Cargo.toml" data-bin="app" data-type="main" data-weak-refs />
+    <link data-trunk rel="rust" href="Cargo.toml" data-bin="worker" data-type="worker" data-weak-refs />


Why not use data-loader-shim here?

This is because .spawn() and .spawn_with_loader()can provide proper tree shaking depending on which function is called. However, I do not think loader={true} will provide proper tree shaking.

I think a potential way to work it to provide it via a trait.
(a.k.a: <WorkerProvider<Codec, ShimLoader> ...> vs <WorkerProvider<Codec, DefaultLoader> ...>)

However, there is no good way of knowing whether tree shaking works unless actually try compiling it after implementing it. So I excluded it from this release.

I also feel it might be OK to require data-shim-loader for all agents given we properly document how the loader shim works.

ranile · 2023-08-29T17:02:49Z

packages/yew-agent-macro/src/lib.rs

+#[proc_macro_attribute]
+pub fn reactor(attr: TokenStream, item: TokenStream) -> TokenStream {
+    let item = parse_macro_input!(item as AgentFn<ReactorFn>);
+    let attr = parse_macro_input!(attr as AgentName);
+
+    reactor_impl(attr, item)
+        .unwrap_or_else(|err| err.to_compile_error())
+        .into()
+}
+
+#[proc_macro_attribute]
+pub fn oneshot(attr: TokenStream, item: TokenStream) -> TokenStream {
+    let item = parse_macro_input!(item as AgentFn<OneshotFn>);
+    let attr = parse_macro_input!(attr as AgentName);
+
+    oneshot_impl(attr, item)
+        .unwrap_or_else(|err| err.to_compile_error())
+        .into()
+}


Why can't the gloo-worker macros be used here?

Because gloo_worker_macros references gloo_worker in generated code. yew-agent-macro needs to reference yew-agent in generated code.

In addition, it is not possible to expose anything other than procedural macros from a macro crate.
A new crate is needed from the gloo side to expose the implementations of procedural macros.

The macro implementation of yew-agent and gloo-worker also differs in diagnostics (agents are named agents instead of workers).

I would prefer to expose gloo-worker-macro-impl crate (that can also be compiled to wasm) and then call it from there. It's fine for now. I presume we'd need this crate this yew-agent-macro either way so I'm fine with merging duplicate implementation of it.

packages/yew-agent/src/oneshot/hooks.rs

packages/yew-agent/src/oneshot/provider.rs

packages/yew-agent/src/reactor/hooks.rs

ranile · 2023-08-29T17:16:40Z

packages/yew-agent/src/reactor/hooks.rs

+
+/// A hook to subscribe to the outputs of a [Reactor] agent.
+///
+/// All outputs sent to current bridge will be collected into a slice.


Why collect them into a slice instead of providing a stream?

There are 2 hooks for reactor and worker agents - use_*_bridge and use_*_subscription.

The use_*_bridge hook uses a callback-like scheme where each output from agents will call the callback function.
The use_*_subscription hook mimics a common useSubscription -style hook from common React GraphQL clients.
For Subscriptions, past messages are collected into a slice. So one can do things like:

let messages_html = messages_data .iter() .map(|m| html! { <div key={m.id}>{m.content}</div> }) .collect::<Vec<_>>();

Another reason to avoid a stream is to avoid a spawn_local.
If a stream is provided, then a future must be spawned to actively poll the stream.

If users do not need things to be collected into a slice, then they can use the _bridge variant with a callback.

I see, that makes sense. It would be good to have an example in the doc comment.

packages/yew-agent/src/reactor/provider.rs

futursolo · 2023-09-02T09:07:56Z

We should have a struct components example too

I think it is fine with without struct component examples.
Just like the tutorial is switched to all function components, we should continue to promote users to use function components.

ranile

I skimmed through the macro code and it looks good to me.

We should have a struct components example too

I think it is fine with without struct component examples. Just like the tutorial is switched to all function components, we should continue to promote users to use function components.

I agree that we should promote users towards function components. we should still absolutely support struct components and provide examples on how to use features with them. I'm fine with merging the PR without it so we're not blocking the release (I would like to get all the 0.21 version s released as soon as possible), we should definitely add those examples later

I'm wondering if there's a way to have an async function that is run inside a worker and the result of it can be .awaited on the main thread?

ranile · 2023-09-11T18:02:31Z

packages/yew-agent-macro/src/lib.rs

+#[proc_macro_attribute]
+pub fn reactor(attr: TokenStream, item: TokenStream) -> TokenStream {
+    let item = parse_macro_input!(item as AgentFn<ReactorFn>);
+    let attr = parse_macro_input!(attr as AgentName);
+
+    reactor_impl(attr, item)
+        .unwrap_or_else(|err| err.to_compile_error())
+        .into()
+}
+
+#[proc_macro_attribute]
+pub fn oneshot(attr: TokenStream, item: TokenStream) -> TokenStream {
+    let item = parse_macro_input!(item as AgentFn<OneshotFn>);
+    let attr = parse_macro_input!(attr as AgentName);
+
+    oneshot_impl(attr, item)
+        .unwrap_or_else(|err| err.to_compile_error())
+        .into()
+}


I would prefer to expose gloo-worker-macro-impl crate (that can also be compiled to wasm) and then call it from there. It's fine for now. I presume we'd need this crate this yew-agent-macro either way so I'm fine with merging duplicate implementation of it.

ranile · 2023-09-11T18:09:56Z

packages/yew-agent/src/reactor/hooks.rs

+
+/// A hook to subscribe to the outputs of a [Reactor] agent.
+///
+/// All outputs sent to current bridge will be collected into a slice.


I see, that makes sense. It would be good to have an example in the doc comment.

futursolo · 2023-09-18T08:28:46Z

I'm wondering if there's a way to have an async function that is run inside a worker and the result of it can be .awaited on the main thread?

Isn't that the oneshot worker / agent?

https://github.com/rustwasm/gloo/blob/master/examples/markdown/src/lib.rs#L5

futursolo added 30 commits March 19, 2022 18:42

Make private bridges.

c764a24

Add worker agent.

80470e6

Add task type.

14b87d1

Memorised Task.

9e09ea5

Add station.

b424a13

Add Station Agent.

d9d9034

Subscription.

c6f5405

Station hooks.

7c96635

Reactor Agents.

da6afa2

Add more description.

5fb8ea6

Restart station when closed.

31475ba

Remove Station restart.

8e27e10

Send finish message to bridges.

381089b

Adds a method to read whether a bridge has received finish message fo…

20ec679

…r a subscription.

Update Reactor Agent.

4271aa3

Decouple macros from it.

43a3925

Reactor Macro.

f6262cf

Reactivate Task.

6a3f4d9

Merge branch 'master' into agent-v2

ece1f32

# Conflicts: # packages/yew-agent/src/hooks.rs

Slightly adjust API.

dab6765

Add documentation for reactor agents.

36277b4

Remove Station.

a7ad43f

Create Task macro.

60bd5d5

Migrate Example.

21e7422

Simplify Task Agent Design.

7b36753

Implement Sink.

3ee1997

Agent -> Task in Example.

c396c0a

Switch to Registrable.

a1ac4c5

Merge branch 'master' into agent-v2

43a844c

AgentScopeExt.

8765abc

futursolo added 12 commits August 20, 2023 20:09

Rename Prime to PrimeReactor.

07680ec

Update Crate Information.

e98d8a3

Remove unused dependencies.

0b8e678

Remove unused dependencies.

b40418c

Update documentation.

a5e6f2c

Rename Bridge to Runner.

2040906

Update documentation.

5a6410e

Update documentation.

2157b75

Update State name.

d086af8

Merge outputs state for subscriptions.

cc1fa1f

Update documentation.

266be2b

Merge branch 'master' into agent-v2

78944c3

futursolo marked this pull request as ready for review August 29, 2023 10:57

github-actions bot previously approved these changes Aug 29, 2023

View reviewed changes

ranile requested changes Aug 29, 2023

View reviewed changes

futursolo added 4 commits September 2, 2023 13:07

Fix doc link.

c74bd6b

Make code link code link.

6f02bb8

Make CODEC -> C.

898de87

Update Debug Implementation to type_name.

3d1b767

futursolo dismissed github-actions[bot]’s stale review via 3d1b767 September 2, 2023 04:15

github-actions bot previously approved these changes Sep 2, 2023

View reviewed changes

Fix readme.

7d84e37

futursolo dismissed github-actions[bot]’s stale review via 7d84e37 September 2, 2023 09:06

github-actions bot approved these changes Sep 2, 2023

View reviewed changes

futursolo requested a review from ranile September 2, 2023 09:19

SpanishPear mentioned this pull request Sep 9, 2023

[ WIP ] Attempted upgrade to yew 0.20 for mipsy-web, Run does not work insou22/mipsy#293

Draft

ranile approved these changes Sep 11, 2023

View reviewed changes

futursolo merged commit eec0758 into yewstack:master Sep 18, 2023
20 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Agent v2 #2773

Agent v2 #2773

futursolo commented Jul 3, 2022 •

edited

Loading

ranile left a comment

ranile Aug 29, 2023

futursolo Sep 2, 2023

ranile Aug 29, 2023

futursolo Sep 2, 2023

ranile Aug 29, 2023

futursolo Sep 2, 2023

ranile Sep 11, 2023

ranile Aug 29, 2023

futursolo Sep 2, 2023

ranile Sep 11, 2023

futursolo commented Sep 2, 2023

ranile left a comment

ranile Sep 11, 2023

ranile Sep 11, 2023

futursolo commented Sep 18, 2023

Agent v2 #2773

Agent v2 #2773

Conversation

futursolo commented Jul 3, 2022 • edited Loading

Description

Questions to solve:

Still missing:

Checklist

ranile left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

futursolo commented Sep 2, 2023

ranile left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

futursolo commented Sep 18, 2023

futursolo commented Jul 3, 2022 •

edited

Loading