If Servers Could Vote: Understanding the Raft Distributed Consensus Algorithm
Distributed systems often need several servers to agree on the same sequence of operations. That agreement is called consensus. It sounds simple when described as "the servers should all have the same data," but achieving it becomes difficult when machines can fail, messages can be delayed, or different servers temporarily believe different things.
Raft is a distributed consensus algorithm designed to make this problem easier to understand and implement. Its central idea is to organize a cluster around a leader. Servers vote to choose that leader, the leader coordinates changes, and a quorum of servers confirms which changes are safe to consider committed.
The "servers could vote" metaphor is useful because Raft treats many important decisions as elections. A server may become a candidate, request votes, and become the leader if it receives enough support. Once elected, the leader uses heartbeats and log replication to keep the cluster aligned.
This article explains the main ideas associated with Raft: cluster composition, roles, terms, leader election, heartbeats, log replication, and quorum. It also shows how these pieces fit together and what practical engineering questions they help answer.
Why Consensus Is Needed
Suppose an application stores important state on multiple servers. Replication can improve availability: if one server fails, another may still have a copy of the data. But replication introduces a coordination problem.
Imagine that a client requests an operation such as:
set balance = 100
If one server accepts the operation while another accepts a conflicting operation, the replicas may diverge. If clients later read from different servers, they may observe different results. The system needs a way to decide which operations belong to the shared history and in what order.
A consensus algorithm provides a structured way for a group of servers to agree on that history. It must continue to make useful progress when failures are limited and must avoid declaring conflicting histories as committed. Raft approaches this by dividing responsibility clearly:
- One server acts as the leader.
- Other servers act as followers.
- Followers can become candidates when they stop hearing from a leader.
- The leader receives operations and replicates them to followers.
- A quorum determines whether an operation has enough support to be considered committed.
This structure does not eliminate failures. Instead, it defines how the cluster should react to them.
The Cluster and Its Server Roles
A Raft cluster contains multiple servers, often called nodes. Each server maintains its own local view of the replicated log and participates in the protocol.
At a given time, a server has one of three primary roles:
Follower
A follower is a passive participant. It responds to requests from the current leader and participates in elections when necessary. A follower normally expects to receive periodic communication from the leader.
If that communication stops for long enough, the follower may conclude that the leader is unavailable. It then changes into the candidate role and starts an election.
Candidate
A candidate is a server attempting to become the new leader. It begins an election by moving to a new term, voting for itself, and asking other servers for votes.
A candidate becomes leader only if it obtains votes from a quorum. If another server is elected, or if the candidate discovers a newer term, it returns to the follower role.
Leader
The leader coordinates normal operation. Clients send changes to the leader, and the leader attempts to replicate those changes to followers. The leader also sends regular heartbeats, which are messages that reassure followers that the leader is still active.
The leader role is temporary. If the leader fails or loses contact with the rest of the cluster, another election can choose a replacement.
At most one leader should be recognized for a particular term within the same effective cluster. Terms and voting rules help servers distinguish current leadership from stale messages or old leaders.
Terms: Logical Periods of Cluster Activity
Raft divides time into numbered logical periods called terms. A term is not necessarily a fixed amount of wall-clock time. It is more like an epoch or generation number used to order leadership attempts and protocol messages.
A term can begin when a server starts an election. The candidate increments the term and asks other servers to vote. If it wins, it becomes leader for that term. If it discovers that another message contains a higher term, it knows that its own information is outdated and steps down to follower.
Terms are useful because networks can delay messages. A message from an old leader might arrive after a new leader has already been elected. Without a way to compare the freshness of messages, a server could accidentally treat stale leadership information as current.
A simple rule is:
A message from a higher term takes precedence over information from a lower term.
This rule allows servers to reject or downgrade outdated participants. Terms do not by themselves solve every consistency problem, but they provide an essential ordering mechanism for elections and replication.
Leader Election
Leader election begins when a follower does not receive expected communication from the leader. The follower waits for an election timeout. If the timeout expires, it becomes a candidate.
The candidate then performs a sequence of actions:
- It increments its current term.
- It changes its role to candidate in the new term.
- It votes for itself.
- It requests votes from other servers.
- It becomes leader if it receives votes from a quorum.
The exact waiting periods used by an implementation are configuration details, but the conceptual purpose is important: followers should not all begin elections at precisely the same moment. If every server becomes a candidate simultaneously, votes may split and no candidate may obtain a quorum. Randomized election timeouts make it more likely that one candidate starts first and gathers support.
What Happens When a Candidate Wins?
After receiving enough votes, the candidate becomes leader. It should begin sending heartbeats to the other servers. These messages serve two purposes:
- They communicate that a leader exists for the current term.
- They help prevent followers from starting unnecessary elections.
The new leader also becomes responsible for coordinating future log replication.
What Happens When Votes Are Split?
A candidate may fail to obtain a quorum because different candidates receive different votes. In that situation, no candidate has enough authority to become leader. The cluster can wait for another election attempt, usually after time advances and a new term begins.
A split vote is different from two candidates being legitimately accepted as leaders by the same quorum. The quorum principle is designed to prevent two conflicting majorities from existing at the same time when majorities overlap.
What Happens When a Candidate Discovers a Newer Term?
If a candidate receives a message showing a higher term, it recognizes that its own term is stale. It returns to follower status and updates its term information. This prevents an old candidate from continuing to act as though it still has a current claim to leadership.
Heartbeats and Failure Detection
A heartbeat is a periodic message from the leader to followers. In Raft, heartbeats are closely connected to leader election. They tell followers that the leader is still communicating with the cluster.
A heartbeat can be viewed as a replication message that contains no new client operation. Its existence is still meaningful: it confirms that the leader is active and gives followers a chance to learn the current term and other protocol information.
If a follower continues receiving heartbeats, it resets its election timer. If heartbeats stop arriving, the follower eventually suspects that the leader is unavailable and starts an election.
This is a form of failure detection, but it should not be interpreted as perfect knowledge. A missing heartbeat may mean that:
- The leader has crashed.
- The follower has crashed and restarted.
- A network path is temporarily unavailable.
- Messages are delayed.
- The cluster is under heavy load.
The protocol reacts to the absence of communication rather than knowing the exact cause. That is why timing configuration matters. Timeouts that are too short may cause needless elections during temporary delays. Timeouts that are too long may delay recovery after a genuine failure.
Heartbeats therefore provide both liveness support and leadership communication. They do not replace log replication; they maintain the relationship between the leader and followers when there may be no new data to replicate.
The Replicated Log
Raft represents the shared sequence of operations as a log. A log is an ordered list of entries. Each entry records an operation that the system may eventually apply to its state machine.
For example, a log might contain entries such as:
1. create account A
2. deposit 50 into account A
3. withdraw 20 from account A
The order is important. Applying the withdrawal before the account exists could produce a different result from applying it after the creation and deposit operations.
The leader is normally the point through which new operations enter the replicated log. When a client sends an operation, the leader adds an entry to its own log and attempts to send that entry to followers. Followers store matching entries when they accept the leader's replication request.
The goal is not merely for each server to contain some copy of the data. The goal is for the servers to agree on a consistent ordered history, especially on the entries that are committed and safe to apply.
Log Replication
A typical replication flow looks like this:
- A client sends an operation to the leader.
- The leader appends the operation to its local log.
- The leader sends the new entry to followers.
- Followers record the entry if it fits their log history.
- The leader determines whether enough servers have accepted the entry.
- Once the entry satisfies the commit condition, the leader considers it committed.
- Servers apply committed entries to their state machines in log order.
This separates several ideas that are easy to confuse:
- Appended means an entry has been placed in a server's local log.
- Replicated means multiple servers have stored the entry.
- Committed means the protocol has enough agreement to treat the entry as part of the durable shared history.
- Applied means a server has processed the committed entry in its state machine.
An entry can be present locally without being committed. For example, the leader could fail immediately after appending an entry but before enough followers receive it. The replacement leader must decide how to handle entries that are not supported by the required consensus evidence.
Keeping Logs Consistent
Followers may have logs that are behind the leader. They may also contain entries from an earlier leadership period. The leader must bring followers toward a compatible history rather than simply adding new entries blindly.
The important consistency idea is that a follower should accept a new log entry only when the preceding history matches the leader's expectation. If the follower's prior entry does not match, the replication attempt fails, and the leader must retry with an earlier position or otherwise reconcile the difference.
This process helps eliminate conflicting suffixes. A follower might have entries that were added during a previous leadership period but never achieved the quorum necessary for commitment. When a current leader establishes a different authoritative history, those uncommitted conflicting entries may be replaced so that the follower's log can match the leader's committed direction.
The exact mechanics of indexes, previous-entry checks, and retry positions are implementation details, but the practical purpose is straightforward: all servers should converge on one compatible sequence, and committed entries must not be silently contradicted.
The Quorum Principle
A quorum is a sufficient number of servers needed to approve or support a decision. In the common majority-based design, a quorum is more than half of the cluster.
For a cluster with five servers, a majority is three. For a cluster with three servers, a majority is two. The key property is that any two majorities overlap in at least one server.
That overlap matters. If one group of servers supports a decision and another group supports a conflicting decision, the groups cannot both be disjoint majorities. At least one server belongs to both groups, creating a connection between the decisions.
Quorums are used in several important situations:
- Electing a leader.
- Confirming that a log entry has enough replication support.
- Preventing two independent groups from both making authoritative decisions under the same conditions.
A larger cluster is not automatically more available. The cluster must still obtain a quorum to elect a leader or commit new operations. If too many servers are unreachable, the remaining servers may be unable to make progress even if some individual machines are healthy.
This is an essential operational trade-off: quorum-based safety may require sacrificing progress when the cluster cannot communicate with enough members.
A Worked Example with Five Servers
Consider a cluster containing five servers: A, B, C, D, and E. A majority is three servers.
Suppose A is the leader. A client sends an operation to A. A appends it to its log and sends it to B and C. If B and C accept it, three servers—A, B, and C—support the entry. That is a quorum, so the leader can treat the entry as committed according to the protocol's commit rules.
Now suppose D and E are temporarily disconnected. The cluster still has three communicating servers, which is enough for a majority. The system may continue making progress, assuming the remaining servers can maintain a valid leadership arrangement.
Next, suppose A, B, and C become unavailable, leaving only D and E. Two servers remain, but two is not a majority of five. They cannot safely elect a leader or commit new operations based only on their local agreement. They may contain useful data, but they do not have enough quorum support to make authoritative progress for the full cluster.
This example shows why "some servers are alive" is not the same as "the cluster can make progress." Consensus depends on sufficient agreement, not merely on isolated machine health.
Safety and Progress
Two broad goals appear repeatedly in consensus discussions: safety and progress.
Safety means the system does not make contradictory decisions. For example, two different values should not both become committed for the same position in the shared history. Log consistency, terms, leader rules, and quorum intersection all contribute to safety.
Progress, sometimes called liveness, means the cluster can continue making decisions when the required conditions are available. Leader election, heartbeats, and retrying replication help the system recover from failures and continue operating.
These goals can pull in different directions. Waiting for more evidence can improve safety but delay progress. Acting quickly can improve responsiveness but risks reacting to temporary network problems. Raft's structure provides rules for making progress while retaining strong consistency properties under its intended failure assumptions.
A practical engineer should therefore ask two separate questions:
- Can the cluster avoid accepting conflicting histories?
- Can the cluster obtain enough communication and votes to continue operating?
A design can be safe but temporarily unavailable when it lacks a quorum. That behavior is not necessarily a defect; it is a consequence of requiring sufficient agreement.
How Raft Handles Common Failure Scenarios
Understanding Raft means recognizing how it responds to different failure modes:
Leader Failure
When the leader crashes, followers stop receiving heartbeats. After the election timeout expires, one follower becomes a candidate and starts an election. If it obtains a quorum of votes, it becomes the new leader and begins sending heartbeats. Uncommitted entries from the old leader may be discarded if they lack sufficient replication support.
Follower Failure
When a follower crashes, the leader continues sending replication messages. The leader may not immediately know that the follower is unavailable, but it will eventually detect the lack of acknowledgment. The leader can still make progress if enough other followers accept the entries to form a quorum.
Network Partition
If the cluster splits into two groups that cannot communicate, the group containing a quorum can continue operating under the current leader or elect a new one. The minority group cannot safely make progress because it cannot obtain a quorum. When the partition heals, the minority group's uncommitted entries may be discarded, and the servers will adopt the committed history from the majority group.
Delayed Messages
Old messages arriving late are handled through term numbers. A message from an old term is ignored or causes the recipient to update to a newer term. This prevents stale leadership information from disrupting the current cluster state.
A Practical Mental Model
Raft can be remembered as a recurring cycle:
- The cluster has a leader.
- The leader sends heartbeats and accepts operations.
- Operations are appended to the leader's log.
- The leader replicates them to followers.
- A quorum supports the operation.
- The operation becomes committed and is applied in order.
- If the leader stops communicating, followers begin an election.
- A candidate obtains a quorum of votes and becomes the next leader.
This model connects the major topics together. Elections establish who coordinates the cluster. Heartbeats maintain that leadership. Log replication distributes the ordered history. Quorums determine whether leadership and changes have enough support.
Practical Engineering Takeaways
When evaluating or explaining a Raft-based system, focus on these questions:
- How many servers are in the cluster?
- How many servers constitute a quorum?
- What causes a follower to start an election?
- How do servers recognize that a message belongs to a newer term?
- How are client operations routed to the leader?
- How are log entries sent to followers?
- What distinguishes an appended entry from a committed entry?
- What happens when a follower's log is behind or conflicts with the leader's log?
- What happens when the cluster loses contact with enough members to form a quorum?
- How are committed entries applied in the same order on each server?
- What is the election timeout, and how does it affect recovery speed?
- How does the leader know which entries are safe to commit?
- What prevents two servers from both claiming to be leader in the same term?
These questions help turn a high-level animation or diagram into an operational understanding of the algorithm.
Implementation Considerations
While Raft's core ideas are conceptually clean, real implementations must address several practical details:
Persistence: Servers must store term numbers, voted-for information, and log entries on stable storage. A server that crashes and restarts must recover this state to maintain safety guarantees.
Timing: Election timeouts and heartbeat intervals must be chosen carefully. Election timeouts are typically randomized to avoid simultaneous elections. Heartbeat intervals should be much shorter than election timeouts to prevent unnecessary elections during normal operation.
Log Compaction: Logs grow over time. Raft systems typically implement snapshotting to discard old log entries that have been applied to the state machine. Snapshots allow new servers to catch up quickly without replaying the entire history.
Client Interaction: Clients must know how to find the leader. Some implementations use a fixed leader address and rely on clients to retry if they contact a follower. Others use service discovery or client libraries that handle leader discovery automatically.
Membership Changes: The basic Raft algorithm assumes a fixed cluster membership. Changing the set of servers requires careful coordination to avoid creating two independent clusters. Extensions to Raft address this by using a two-phase approach to membership changes.
Conclusion
Raft makes distributed consensus easier to reason about by giving the cluster a clear organization. Servers have roles, leadership is established through votes, terms identify logical periods of activity, heartbeats maintain contact, and the leader coordinates log replication.
The quorum principle is the foundation that ties these mechanisms together. A leader needs enough votes to be recognized, and a log entry needs enough support to become part of the committed history. Because majorities overlap, the cluster can avoid treating two conflicting groups as independently authoritative under the same conditions.
The most useful summary is simple: servers vote to choose a leader, the leader maintains an ordered log, heartbeats preserve leadership, and a quorum provides the agreement needed to make decisions safely. Once these ideas are understood, Raft's cluster composition, elections, replication process, and failure behavior become parts of one coherent system rather than isolated protocol features.
Raft has become a popular choice for building fault-tolerant distributed systems because it achieves a good balance between safety, understandability, and practical implementability. By understanding how its components work together—how elections establish leadership, how heartbeats maintain it, how log replication distributes state, and how quorums ensure agreement—engineers can build and reason about systems that reliably coordinate multiple machines even when failures occur.