consistent query bug

[kjnilsson]

Describe the bug

https://github.com/rabbitmq/ra-kv-store/actions/runs/12121409210

In. one test (in a long history of non failing runs) one leader in a minority partition decided it was ok to serve consistent reads.
After a fair bit of time investigating it wasn't clear exactly what caused it but I suspect it was due to the leader somehow ending up with an incorrectly high query_index for one of it's peers.

The consistent query logic can be refactored to avoid relying on this piece of state entirely as all it does is perform a leader liveness check then waits for the commit index to be appled then execute the query.

This check can be done without each member keeping a query_index and instead just returning the {term, index} tuple provided if the term is equal or higher.

Reproduction steps

https://github.com/rabbitmq/ra-kv-store/actions/runs/12121409210

Expected behavior

.

Additional context

handle_follower(#heartbeat_rpc{query_index = RpcQueryIndex, term = Term,
                               leader_id = LeaderId},
                #{current_term := CurTerm,
                  cfg := #cfg{id = Id}} = State0)
  when Term >= CurTerm ->
    State1 = update_term(Term, State0),
    #{query_index := QueryIndex} = State1,
    NewQueryIndex = max(RpcQueryIndex, QueryIndex),
    State2 = update_query_index(State1#{leader_id => LeaderId}, NewQueryIndex),
    Reply = heartbeat_reply(State2),
    {follower, State2, [cast_reply(Id, LeaderId, Reply)]};

In this code, if the follower somehow ended up with a local_query index that was higher than what the leader has (each term resets the query index) then that could cause a leader to locally store a higher query index which if this follower was later partition would still contribute to the quorum calculation. If we can remove this state we can remove this possibility.