Skip to content

Commit

Permalink
some typo and style update to the Readme.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@lijunwangs
lijunwangs committed Dec 19, 2024
1 parent a4b63f0 commit 6125f44
Showing 1 changed file with 75 additions and 115 deletions.
190 changes: 75 additions & 115 deletions vortexor/Readme.md
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Introduction
The vortexor is a service which can be used to offload receiving transaction
from the public, performing signature verifications and deduplications from the
core validator enabling it to focus on processing and executing the
transactions. The verified and filtered transactions then will be forwarded to
the validators linked with the vortexor. This vortexor makes the TPU transaction
ingestion and verification more scalable compared with the single node solution.
The Vortexor is a service that can offload the tasks of receiving transactions
from the public, performing signature verifications, and deduplications from the
core validator, enabling it to focus on processing and executing the
transactions. The verified and filtered transactions will then be forwarded to
the validators linked with the Vortexor. This setup makes the TPU transaction
ingestion and verification more scalable compared to a single-node solution.

# Archietecure
Figure 1 describes the archietecture diagram of the Vorexor with the
relationship to the validator.
# Architecture
Figure 1 describes the architecture diagram of the Vortexor and its
relationship with the validator.

+---------------------+
| Solana |
Expand All @@ -35,7 +35,7 @@ relationship to the validator.
| +----------------+ | |
+--------------------------------|----------------------+
^ | (UDP/QUIC)
heart beat/subscriptions | |
Heartbeat/subscriptions | |
| v
+-------------------- AGAVE VALIDATOR ------------------+
| |
Expand All @@ -53,40 +53,33 @@ relationship to the validator.

Figure 1.

The Vorexor is a new executable which can be deployed on to different nodes from
the core Agave validator. It can also be deployed onto the same node as the core
validator if the node has enough performance bandwidth.
The Vortexor is a new executable that can be deployed on nodes separate from
the core Agave validator. It can also be deployed on the same node as the core
validator if the node has sufficient performance bandwidth.

It has the following major components:

1. The TPU Streamer -- this is built from the existing QUIC based TPU streamer
2. The SigVerify/Dedup -- this is built/refactored from the existing SigVerify
component
3. Subscription Management -- This is responsible for managing subscriptions
from the validator. Subscriptions action include subscription for
transactions and cancel subscriptions.
4. VerifiedPacketForwarder -- This is responsible for forwarding the verified
transaction packets to the subscribed validators. We target use UDP/QUIC to
send transactions to the validators. The validator has option to bind to
private address for receiving the verified packets.
The validators can also use firewall rules to allow transactions only from
the chosen vortexor.
5. The Vortexor StakedKeyUpdater -- this service is responsible for retrieving
the stake map from the network and make it available to the TPU streamer
so that it can apply stake-weighted QOS.

In the validator, there is new component which receives the verified packets
sent from the vortexor which directly sends the packets to the banking stage.
The validator's Admin RPC is enhanced to configure the peering vortexor. The
ContactInfo of the validator is updated with the address of the vortexor when it
is linked with the validator.
1. **The TPU Streamer** – This is built from the existing QUIC-based TPU streamer.
2. **The SigVerify/Dedup** – This is refactored from the existing SigVerify component.
3. **Subscription Management** – Responsible for managing subscriptions
from the validator. Actions include subscribing to transactions and canceling subscriptions.
4. **VerifiedPacketForwarder** – Responsible for forwarding verified
transaction packets to subscribed validators. It uses UDP/QUIC to send transactions.
Validators can bind to private addresses for receiving the verified packets.
Firewalls can also restrict transactions to the chosen Vortexor.
5. **The Vortexor StakedKeyUpdater** – Retrieves the stake map from the network and makes
it available to the TPU streamer for stake-weighted QoS.

Validators include a new component that receives verified packets sent from
the Vortexor and directly sends them to the banking stage. The validator's
Admin RPC is enhanced to configure peering with the Vortexor. The ContactInfo of
the validator updates with the Vortexor's address when linked.

# Relationship of Validator and Vortexor
The validator always broadcast one TPU address which will be served by a
vortexor. A validator can change its pairing vortexor to another. A vortexor
based on its performance can serve 1 or more validators. The architecture
also allows multiple vortexors sharing the TPU address behind a load balancer
to serve a validator to make the solution more scalable -- see blow.
The validator broadcasts one TPU address served by a Vortexor. A validator can
switch its paired Vortexor to another. A Vortexor, depending on its performance,
can serve one or more validators. The architecture also supports multiple
Vortexors sharing the TPU address behind a load balancer for scalability:

Load Balancer
|
Expand All @@ -104,81 +97,48 @@ to serve a validator to make the solution more scalable -- see blow.

Figure 2.

When the validator is in the 'Paired' mode which it is either getting active
transactions from the corresponding vortexor or receiving heartbeat messages,
the validator solely receive TPU transactions from the vorexor alone as it only
publishes the TPU address via gossip. Further, the regular TPU and TPU forward
services in the validator will be put into the disabled mode for security and
performance reasons.

It is the presumption of this design that there is a trust between the vortexor
and the validator. The trust can be achieved either by placing them into the
same private network, using firewall rules plus TLS verification. When using
QUIC on as transport for the VerifiedPacketReceiver on the validator, the
validator can still set the QOS against the vortexor. And it is expected the
validator will grant much higher bandwidth to the vortexor compared with regular
TPU clients in a regular validator's TPU configurations.

There is periodic heartbeat messages sent from the vortexor to the validator.
If there are not transactions sent and no heartbeat messages from the vortexor
within configurable timeout window, the validator may decide the vortexor is
dead or disconnected it may choose to use another vortexor or use its own built-
in QUIC-based TPU streamer by updating the ContactInfo about its TPU address.
When the validator is in 'Paired' mode, receiving active transactions or
heartbeat messages from the Vortexor, it receives TPU transactions solely from
the Vortexor. It publishes the TPU address via gossip. The regular TPU and TPU
forward services are disabled for security and performance reasons.

The design assumes a trust relationship between the Vortexor and the validator,
achieved through a private network, firewall rules, or TLS verification. QUIC,
used for the VerifiedPacketReceiver, supports QoS to prioritize Vortexor traffic.

Heartbeat messages from the Vortexor inform the validator of its status. If no
transactions or heartbeats are received within a configurable timeout, the
validator may switch to another Vortexor or revert to its built-in TPU streamer.

# Deployment Considerations
The vortexor while making the validator more scalable in handling transactions,
does have its drawbacks:

1. It increases the deployment complexity. By default, for validators which
do not use vortexors, there is no deployment and functional/performance
impacts. For validators using vortexors, it requires addtional deployment task
to deploy the vortexor. For performance considerations, it is most likely the
vortexor will be running in a seperate node from its pairing validator. To
mitigate complexity of the deployment, the vortexor will support taking minimal
arguments to work with the validator, and similarly it takes minimal changes
on the validator to work with the vortexor. There will be clear public
documentation how to run a vortexor and to pair a validator with it. There
will be auto fallback mechanism built-in that in the case of conneciton breakage
between the vortexor and the validator, the validator auto fallback to its
built-in streamer. In addition, the validator and the vortexor will support the
additional admin RPC to query the vortexor pairing states and mange pairing
relationship.

2. There is an extra hop from the original clients sending the transaction to
the leader validator. This is a trade-off between scalability and latency. The
latency can be minimized as the vortexor is expected to run on a node which
is on the private network with low latency to the validator. With this
consideration, the solution also supports pure UDP to forward transactions
from the vortexor to the validator.

3. The security implications, there is an implict trust relationship between
the validator and the vortexor. It is expected that the vortexor and the
validator to be running in the same private network. In addition, firewall rules
can be used to limit access to the validator's VerifiedPacketReceiver's port
only to the authorized validator. We also support using QUIC as the transport.
In the QUIC we can have rule to limit the connections from the known pubkey.
Finally, there can be an option to enforce the transaction to go through
sigverifications in the validator (default will be off due to its added
computing cost -- double verifications).

4. There is already some solution like jito-relayer being used by validators.
The solutions will be compatibile with jito-relayers in that it should not
impact validators already using jito validator and relayers. Also, we will keep
the arguments of vortexor CLI close to jito-relayer's CLI as possible to reduce
surprises for validators migrating to using vortexor from the jito-relayer.

5. The vortexor's networking setup. In the simplest format, The vortexor's
TPU and TPU forward port might be directly accessible to the internet. Or it
might be put behind the load balancer for security and performance.
The vortexor is encouraged to communicate with the validator using a private
network for performance and security considerations.

# Uprade Considerations
It is up to the operators to decide if to adopt vortexors. Operators can either
use the vortexor or not use it without concerns if the rest of the network's
decision as the vorexor itself does not change the protocol of the network. When
upgrading an existing validator to use the vortexor, the operator can simpily
update the validator's CLI to specify the vortexor's TPU address and specify the
verified packet receiver's network address. The vortexor can be started with
the pairing core validator's verified packet receiver address. These can also be
achieved by the respective Admin RPC on both of the valdiator and the vortexor.
Using a Vortexor enhances validator scalability but introduces complexities:

1. **Deployment Complexity**: For validators not using a Vortexor, there is no
impact. For those using a Vortexor, additional setup is required. To minimize
complexity, the Vortexor and validator require minimal configuration changes
and provide clear documentation for pairing. Automatic fallback ensures
continued operation if the connection between the Vortexor and validator
breaks.

2. **Latency**: An additional hop exists between the original client and the
leader validator. Latency is minimized by deploying the Vortexor on a node
with low-latency connections to the validator. UDP forwarding is supported
for speed.

3. **Security**: The implicit trust between the validator and Vortexor is
safeguarded by private networks, firewalls, and QUIC with public key-based
rules. Validators can optionally enforce re-verification of transactions.

4. **Compatibility**: The solution is compatible with existing setups, such as
jito-relayers. The Vortexor CLI mimics jito-relayer's CLI to reduce friction
for migrating users.

5. **Networking**: The Vortexor can be exposed directly to the internet or
placed behind a load balancer. Communication with the validator is
encouraged via private networks for security and performance.

# Upgrade Considerations
Operators can decide whether to adopt Vortexors without concerns about network
protocol changes. Upgrading involves specifying the Vortexor's TPU address and
verified packet receiver network address via CLI or Admin RPC. The transition is
designed to be seamless for operators.

0 comments on commit 6125f44

Please sign in to comment.