Merge pull request #11 from andreev-io/ilya/publish

Crate ready

README.md

@@ -1,2 +1,134 @@
-# Raft Distributed Consensus Library
+# Little Raft
+The lightest distributed consensus library. Run your own replicated state machine! :heart:
 
+## Using
+To use this library, you only need to do three things.
+
+1. Implement the StateMachine that you want your cluster to maintain. Little Raft will take care of replicating this machine across the cluster and achieving consensus on its state.
+```rust
+/// StateMachine describes a user-defined state machine that is replicated
+/// across the cluster. Raft can Replica whatever distributed state machine can
+/// implement this trait.
+pub trait StateMachine<T>
+where
+    T: StateMachineTransition,
+{
+    /// This is a hook that the local Replica will call each time the state of a
+    /// particular transition changes. It is up to the user what to do with that
+    /// information.
+    fn register_transition_state(&mut self, transition_id: T::TransitionID, state: TransitionState);
+
+    /// When a particular transition is ready to be applied, the Replica will
+    /// call apply_transition to apply said transition to the local state
+    /// machine.
+    fn apply_transition(&mut self, transition: T);
+
+    /// This function is used to receive transitions from the user that need to
+    /// be applied to the replicated state machine. Note that while all Replicas
+    /// poll get_pending_transitions periodically, only the Leader Replica
+    /// actually processes them. All other Replicas discard pending transitions.
+    /// get_pending_transitions must not return the same transition twice.
+    fn get_pending_transitions(&mut self) -> Vec<T>;
+}
+```
+
+2. Implement the Cluster abstraction so that the local Replica can communicate with other nodes.
+```rust
+/// Cluster is used for the local Raft Replica to communicate with the rest of
+/// the Raft cluster. It is up to the user how to abstract that communication.
+/// The Cluster trait also contains hooks which the Replica will use to inform
+/// the crate user of state changes.
+pub trait Cluster<T>
+where
+    T: StateMachineTransition,
+{
+    /// This function is used to deliver messages to target Replicas. The
+    /// Replica will provide the to_id of the other Replica it's trying to send
+    /// its message to and provide the message itself. The send_message
+    /// implementation must not block but is allowed silently fail -- Raft
+    /// exists to achieve consensus in spite of failures, after all.
+    fn send_message(&mut self, to_id: usize, message: Message<T>);
+
+    /// This function is used by the Replica to receive pending messages from
+    /// the cluster. The receive_messages implementation must not block and must
+    /// not return the same message more than once.
+    fn receive_messages(&mut self) -> Vec<Message<T>>;
+
+    /// By returning true from halt you can signal to the Replica that it should
+    /// stop running.
+    fn halt(&self) -> bool;
+
+    /// This function is a hook that the Replica uses to inform the user of the
+    /// Leader change. The leader_id is an Option<usize> because the Leader
+    /// might be unknown for a period of time. Remember that only Leaders can
+    /// process transitions submitted by the Raft users, so the leader_id can be
+    /// used to redirect the requests from non-Leader nodes to the Leader node.
+    fn register_leader(&mut self, leader_id: Option<ReplicaID>);
+}
+```
+3. Start your replica!
+```rust
+    /// Create a new Replica.
+    ///
+    /// id is the ID of this Replica within the cluster.
+    ///
+    /// peer_ids is a vector of IDs of all other Replicas in the cluster.
+    ///
+    /// cluster represents the abstraction the Replica uses to talk with other
+    /// Replicas.
+    ///
+    /// state_machine is the state machine that Raft maintains.
+    ///
+    /// noop_transition is a transition that can be applied to the state machine
+    /// multiple times with no effect.
+    ///
+    /// heartbeat_timeout defines how often the Leader Replica sends out
+    /// heartbeat messages.
+    ///
+    /// election_timeout_range defines the election timeout interval. If the
+    /// Replica gets no messages from the Leader before the timeout, it
+    /// initiates an election.
+    ///
+    /// In practice, pick election_timeout_range to be 2-3x the value of
+    /// heartbeat_timeout, depending on your particular use-case network latency
+    /// and responsiveness needs. An election_timeout_range / heartbeat_timeout
+    /// ratio that's too low might cause unwarranted re-elections in the
+    /// cluster.
+    pub fn new(
+        id: ReplicaID,
+        peer_ids: Vec<ReplicaID>,
+        cluster: Arc<Mutex<C>>,
+        state_machine: Arc<Mutex<S>>,
+        noop_transition: T,
+        heartbeat_timeout: Duration,
+        election_timeout_range: (Duration, Duration),
+    ) -> Replica<S, T, C>;
+
+    /// This function starts the Replica and blocks forever.
+    ///
+    /// recv_msg is a channel on which the user must notify the Replica whenever
+    /// new messages from the Cluster are available. The Replica will not poll
+    /// for messages from the Cluster unless notified through recv_msg.
+    ///
+    /// recv_msg is a channel on which the user must notify the Replica whenever
+    /// new messages from the Cluster are available. The Replica will not poll
+    /// for messages from the Cluster unless notified through recv_msg.
+    ///
+    /// recv_transition is a channel on which the user must notify the Replica
+    /// whenever new transitions to be processed for the StateMachine are
+    /// available. The Replica will not poll for pending transitions for the
+    /// StateMachine unless notified through recv_transition.
+    pub fn start(&mut self, recv_msg: Receiver<()>, recv_transition: Receiver<()>);
+```
+
+
+With that, you're good to go. We are working on examples, but for now you can look at the `little_raft/tests` directory. We're working on adding more tests.
+
+
+## Testing
+Run `cargo test`.
+
+## Contributing
+Contributions are very welcome! Do remember that one of the goals of this library is to be as small and simple as possible. Let's keep the code in `little_raft/src` under 1,000 lines. No PRs breaking this rule will be merged.
+
+You are welcome to pick up and work on any of the issues open for this project. Or you can submit new issues if anything comes up from your experience using this library.

little_raft/Cargo.toml

@@ -1,8 +1,15 @@
 [package]
+description = "The lightest distributed consensus library. Run your own replicated state machine!"
 name = "little_raft"
-version = "0.1.0"
-authors = ["ilya <[email protected]>"]
+version = "0.1.2"
+authors = ["Ilya Andreev <[email protected]>"]
 edition = "2018"
+license = "MIT"
+homepage = "https://github.com/andreev-io/little-raft"
+repository = "https://github.com/andreev-io/little-raft"
+readme = "../README.md"
+keywords = ["distributed-systems", "raft", "consensus"]
+categories = ["concurrency", "database", "database-implementations"]
 
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 

little_raft/src/cluster.rs

@@ -1,23 +1,33 @@
-use crate::{message::Message, state_machine::StateMachineTransition};
-use std::time::Duration;
+use crate::{message::Message, replica::ReplicaID, state_machine::StateMachineTransition};
 
-// Cluster provides the means of communication of a replica with the rest of the
-// cluster and the user.
+/// Cluster is used for the local Raft Replica to communicate with the rest of
+/// the Raft cluster. It is up to the user how to abstract that communication.
+/// The Cluster trait also contains hooks which the Replica will use to inform
+/// the crate user of state changes.
 pub trait Cluster<T>
 where
     T: StateMachineTransition,
 {
-    // This function is used to deliver messages to target replicas. The
-    // algorithm assumes that send can silently fail.
-    fn send(&self, to_id: usize, message: Message<T>);
-    // This function is used to received messages for the replicas. This
-    // function must block until timeout expires or a message is received,
-    // whichever comes first.
-    fn receive_timeout(&self, timeout: Duration) -> Option<Message<T>>;
-    // This function is used to receive actions from the user that the
-    // distributed state machine needs to replicate and apply. All replicas poll
-    // this function periodically but only Leaders merit the return value.
-    // Non-Leaders ignore the return value of get_action.
-    fn get_pending_transitions(&self) -> Vec<T>;
-    fn register_leader_change(&mut self, leader_id: Option<usize>);
+    /// This function is used to deliver messages to target Replicas. The
+    /// Replica will provide the to_id of the other Replica it's trying to send
+    /// its message to and provide the message itself. The send_message
+    /// implementation must not block but is allowed silently fail -- Raft
+    /// exists to achieve consensus in spite of failures, after all.
+    fn send_message(&mut self, to_id: usize, message: Message<T>);
+
+    /// This function is used by the Replica to receive pending messages from
+    /// the cluster. The receive_messages implementation must not block and must
+    /// not return the same message more than once.
+    fn receive_messages(&mut self) -> Vec<Message<T>>;
+
+    /// By returning true from halt you can signal to the Replica that it should
+    /// stop running.
+    fn halt(&self) -> bool;
+
+    /// This function is a hook that the Replica uses to inform the user of the
+    /// Leader change. The leader_id is an Option<usize> because the Leader
+    /// might be unknown for a period of time. Remember that only Leaders can
+    /// process transitions submitted by the Raft users, so the leader_id can be
+    /// used to redirect the requests from non-Leader nodes to the Leader node.
+    fn register_leader(&mut self, leader_id: Option<ReplicaID>);
 }

little_raft/src/lib.rs

@@ -1,5 +1,19 @@
+//! This crate is a small but full-featured implementation of the Raft
+//! distributed consensus protocol. By using this library, you can run a
+//! replicated state machine in your own cluster. The cluster could be comprised
+//! of dozens of physical servers in different parts of the world or of two
+//! threads on a single CPU.
+//!
+//! The goal of this library is to provide a generic implementation of the
+//! algorithm that the library user can leverage in their own way. It is
+//! entirely up to the user how to configure the Raft cluster, how to ensure
+//! communication between the nodes, how to process client's messages, how to do
+//! service discovery, and what kind of state machine to replicate.
+//!
+//! The implementation is kept as simple as possible on purpose, with the entire
+//! library code base fitting in under 1,000 lines of code.
 pub mod cluster;
-mod heartbeat_timer;
 pub mod message;
 pub mod replica;
 pub mod state_machine;
+mod timer;

little_raft/src/message.rs

@@ -1,11 +1,10 @@
 use crate::replica::ReplicaID;
 use crate::state_machine::StateMachineTransition;
 
-// Entry describes a user-defined transition of the distributed state machine.
-// It has some associated metadata, namely the term when the entry was created
-// and its index in the log.
-#[derive(Clone, Debug)]
-pub struct Entry<T>
+/// LogEntry is a state machine transition along with some metadata needed for
+/// Raft.
+#[derive(Clone, Debug, Copy, PartialEq, Eq, PartialOrd)]
+pub struct LogEntry<T>
 where
     T: StateMachineTransition,
 {
@@ -14,41 +13,45 @@ where
     pub term: usize,
 }
 
-// Message describes messages that the replicas pass between each other to
-// achieve consensus on the distributed state machine.
-#[derive(Debug)]
+/// Message describes messages that the replicas pass between each other to
+/// achieve consensus on the distributed state machine.
+#[derive(Debug, Clone, PartialEq, Eq, PartialOrd)]
 pub enum Message<T>
 where
     T: StateMachineTransition,
 {
-    // AppendEntryRequest is used by Leaders to send out logs for other replicas
-    // to append to their log. It also has information on what logs are ready to
-    // be applied to the state machine. AppendEntryRequest is also used as a
-    // heart beat message by Leaders even when no new logs need to be processed.
+    /// AppendEntryRequest is used by the Leader to send out logs for other
+    /// replicas to append to their log. It also has information on what logs
+    /// are ready to be applied to the state machine. AppendEntryRequest is also
+    /// used as a heart beat message by the Leader even when no new logs need to
+    /// be processed.
     AppendEntryRequest {
         from_id: ReplicaID,
         term: usize,
         prev_log_index: usize,
         prev_log_term: usize,
-        entries: Vec<Entry<T>>,
+        entries: Vec<LogEntry<T>>,
         commit_index: usize,
     },
-    // AppendEntryResponse is used by replicas to respond to AppendEntryRequest
-    // messages.
+
+    /// AppendEntryResponse is used by replicas to respond to AppendEntryRequest
+    /// messages.
     AppendEntryResponse {
         from_id: ReplicaID,
         term: usize,
         success: bool,
         last_index: usize,
     },
-    // VoteRequest is used by Candidates to solicit votes for themselves.
+
+    /// VoteRequest is used by Candidates to solicit votes for themselves.
     VoteRequest {
         from_id: ReplicaID,
         term: usize,
         last_log_index: usize,
         last_log_term: usize,
     },
-    // VoteResponse is used by replicas to respond to VoteRequest messages.
+
+    /// VoteResponse is used by replicas to respond to VoteRequest messages.
     VoteResponse {
         from_id: ReplicaID,
         term: usize,