Cleanup spec

docs/spec/consensus/amortized-proving.md

@@ -0,0 +1,63 @@
+# Amortized KZG Prover Backend
+
+It is important that the encoding and commitment tasks are able to be performed in seconds and that the dominating complexity of the computation is nearly linear in the degree of the polynomial. This is done using algorithms based on the Fast Fourier Transform (FFT).
+
+
+This document describes how the KZG-FFT encoder backend implements the `Encode(data [][]byte, params EncodingParams) (BlobCommitments, []*Chunk, error)` interface, which 1) transforms the blob into a list of `params.NumChunks` `Chunks`, where each chunk is of length `params.ChunkLength` 2) produces the associated polynomial commitments and proofs.
+
+We will also highlight the additional constraints on the Encoding interface which arise from the KZG-FFT encoder backend.
+
+## Deriving the polynomial coefficients and commitment
+
+As described in the [Encoding Module Specification](../spec/protocol-modules/storage/encoding.md), given a blob of data, we convert the blob to a polynomial $p(X) = \sum_{i=0}^{m-1} c_iX^i$ by simply slicing the data into a string of symbols, and interpreting this list of symbols as the tuple $(c_i)_{i=0}^{m-1}$.
+
+In the case of the KZG-FFT encoder, the polynomial lives on the field associated with the BN-254 elliptic curve, which as order [TODO: fill in order].
+
+Given this polynomial representation, the KZG commitment can be calculated as in [KZG polynomial commitments](https://dankradfeist.de/ethereum/2020/06/16/kate-polynomial-commitments.html).
+
+
+## Polynomial Evaluation with the FFT
+
+In order to use a Discrete Fourier Transform (DFT) to evaluate a polynomial, the indices of the polynomial evaluations which will make up the Chunks must be members of a cyclic group, which we will call $S$. A cyclic group is the group generated by taking all of the integer powers of some generator $v$, i.e., $\{v^k | k \in \mathbb{Z} \}$  (For this reason, the elements of a cyclic group $S$ of order $|S|=m$ will sometimes be referred to as the $|m|$’th roots of unity). Notice that since our polynomial lives on the BN254 field, the group $S$ must be a subgroup of that field (i.e. all if its elements must lie within that field).
+
+Given a cyclic group $S$ of order $m$, we can evaluate a polynomial $p(X)$ of order $n$ at the indices contained in $S$ via the DFT,
+
+$$
+p_k = \sum_{i=1}^{n}c_i (v^k)^i
+$$
+
+where $p_k$ gives the evaluation of the polynomial at $v^k \in S$. Letting $c$ denote the vector of polynomial coefficients and $p$ the vector of polynomial evaluations, we can use the shorthand $p = DFT[c]$. The inverse relation also holds, i.e., $c = DFT^{-1}[p]$.
+
+To evaluate the DFT programmatically, we want $m = n$. Notice that we can achieve this when $m > n$ by simply padding $c$ with zeros to be of length $m$.
+
+The use of the FFT can levy an additional requirement on the size of the group $S$. In our implementation, we require the size of $S$ to be a power of 2. For this, we can make use of the fact that the prime field associated with BN-254 contains a subgroup of order $2^{28}$, which in turn contains subgroups of orders spanning every power of 2 less than $2^{28}$.
+
+
+As the encoding interface calls for the construction of `NumChunks` Chunks of length `ChunkLength`, our application requires that $S$ be of size `NumChunks*ChunkLength`, which in turn must be a power of 2.
+
+## Amortized Multireveal Proof Generation with the FFT
+
+The construction of the multireveal proofs can also be performed using a DFT (as in [“Fast Amortized Kate Proofs”](https://eprint.iacr.org/2023/033.pdf)). Leaving the full details of this process to the referenced document, we describe here only 1) the index-assignment the scheme used by the amortized multiproof generation approach and 2) the constraints that this creates for the overall encoder interface.
+
+Given the group $S$ corresponding to the indices of the polynomial evaluations and a cyclic group $C$ which is a subgroup of $S$, the cosets of $C$ in $S$ are given by
+
+$$
+s+C = \{g+c : c \in C\} \text{ for } s \in S.
+$$
+
+Each coset $s+C$ has size $|C|$, and there are $|S|/|C|$ unique and disjoint cosets.
+
+Given a polynomial $p(X)$ and the groups $S$ and $C$, the Amortized Kate Proofs approach generates $|S|/|C|$ different KZG multi-reveal proofs, where each proof is associated with the evaluation of $p(X)$ at the indices contained in a single coset $sC$ for $s \in S$. Because the Amortized Kate Proofs approach uses the FFT under the hood, $C$ itself must have an order which is a power of 2.
+
+For the purposes of the KZG-FFT encoder, this means that we must choose $S$ to be of size `NumChunks*ChunkLength` and $C$ to be of size `ChunkLength`, each of which must be powers of 2.
+
+
+## Worked Example
+
+As a simple illustrative example, suppose that  `AssignmentCoordinator` provides the following parameters in order to meet the security requirements of given blob:
+- `ChunkLength` = 3
+- `NumChunks` = 4
+
+Supplied with these parameters, `Encoder.ParamsFromMins` will upgrade `ChunkLength` to the next highest power of 2, i.e., `ChunkLength` = 4, and leave `NumChunks` unchanged. The following figure illustrates how the indices will be assigned across the chunks in this scenario.
+
+![Worked example of chunk indices for ChunkLength=4, NumChunks=4](../../assets/encoding-groups.png)

docs/spec/protocol-modules/storage/assignment.md → docs/spec/consensus/assignment.md

@@ -1,41 +1,22 @@
+## Assignment Module
 
-# Assignment
+The assignment module is essentially a rule which takes in the Ethereum chain state and outputs an allocation of chunks to DA operators. This can be generalized to a function that outputs a set of valid allocations.
 
-The assignment functionality within EigenDA is carried out by the `AssignmentCoordinator`, which is responsible for taking the current OperatorState and the security requirements represented by a given QuorumParam and determining or validating system parameters that will satisfy these security requirements given the OperatorStates. There are two classes of parameters that must be determined or validated:
+A chunk assignment has the following parameters: 
+1) **Indices**: the chunk indices that will be assigned to each DA node. Some DA nodes receive more than one chunk.
+2) **ChunkLength**: the length of each chunk (measured in number of symbols, as defined by the encoding module). We currently require all chunks to be of the same length, so this parameter is a scalar. 
 
-1) the chunk indices that will be assigned to each DA node.
-2) the length of each chunk (measured in number of symbols). In keeping with the constraint imposed by the Encoding module, all chunks must have the same length, so this parameter is a scalar.
+The assignment module is implemented by the `AssignmentCoordinator` interface. 
 
+![image](../../assets/assignment-module.png)
 
-## Interface
+### Assignment Logic
 
-The AssignmentCoordinator must implement the following interface, which facilitates with the above tasks:
+The standard assignment coordinator implements a very simple logic for determining the number of chunks per node and the chunk length, which we describe here.
 
-```go
-type AssignmentCoordinator interface {
+**Chunk Length**
 
-	// GetAssignments calculates the full set of node assignments.
-	GetAssignments(state *OperatorState, blobLength uint, info *BlobQuorumInfo) (map[OperatorID]Assignment, AssignmentInfo, error)
-
-	// GetOperatorAssignment calculates the assignment for a specific DA node
-	GetOperatorAssignment(state *OperatorState, header *BlobHeader, quorum QuorumID, id OperatorID) (Assignment, AssignmentInfo, error)
-
-	// ValidateChunkLength validates that the chunk length for the given quorum satisfies all protocol requirements
-	ValidateChunkLength(state *OperatorState, header *BlobHeader, quorum QuorumID) (bool, error)
-
-	// CalculateChunkLength calculates the chunk length for the given quorum that satisfies all protocol requirements
-	CalculateChunkLength(state *OperatorState, blobLength uint, param *SecurityParam) (uint, error)
-}
-```
-
-## Standard Assignment Security Logic
-
-The standard assignment coordinator implements a very simple logic for determining the number of chunks per node and the chunk length, which we describe here. More background concerning this design can be found in the [Design Document](../../../design/assignment.md)
-
-
-**Chunk Length**.
-
-The protocol requires that chunk lengths are sufficiently small that operators with a small proportion of stake are able to receive a quantity of data commensurate with their stake share. For each operator $i$, let $S_i$ signify the amount of stake held by that operator. 
+Chunk lengths must be sufficiently small that operators with a small proportion of stake will be able to receive a quantity of data commensurate with their stake share. For each operator $i$, let $S_i$ signify the amount of stake held by that operator. 
 
 We require that the chunk size $C$ satisfy
 
@@ -44,15 +25,13 @@ C \le \text{NextPowerOf2}\left(\frac{B}{\gamma}\max\left(\frac{\min_jS_j}{\sum_j
 $$
 
 
-where $\gamma = \beta-\alpha$, with $\alpha$ and $\beta$ as defined in the [Storage Overview](./overview.md).
+where $\gamma = \beta-\alpha$, with $\alpha$ and $\beta$ the adversary and quorum thresholds as defined in the [Overview](../overview.md).
 
 This means that as long as an operator has a stake share of at least $1/M_\text{max}$, then the encoded data that they will receive will be within a factor of 2 of their share of stake. Operators with less than $1/M_\text{max}$ of stake will receive no more than a $1/M_\text{max}$ of the encoded data. $M_\text{max}$ represents the maximum number of chunks that the disperser can be required to encode per blob. This limit is included because proving costs scale somewhat super-linearly with the number of chunks. 
 
 In the future, additional constraints on chunk length may be added; for instance, the chunk length may be set in order to maintain a fixed number of chunks per blob across all system states. Currently, the protocol does not mandate a specific value for the chunk length, but will accept the range satisfying the above constraint. The `CalculateChunkLength` function is provided as a convenience function that can be used to find a chunk length satisfying the protocol requirements. 
 
-
-
-**Index Assignment**.
+**Index Assignment**
 
 For each operator $i$, let $S_i$ signify the amount of stake held by that operator. We want for the number of chunks assigned to operator $i$ to satisfy
 
@@ -66,8 +45,8 @@ $$
 m_i = \text{ceil}\left(\frac{B S_i}{C\gamma \sum_j S_j}\right)\tag{1}
 $$
 
-**Correctness**.
-Let's show that any sets $U_q$ and $U_a$ satisfying the constraints in the [Acceptance Guarantee](./overview.md#acceptance-guarantee), the data held by the operators $U_q \setminus U_a$ will constitute an entire blob. The amount of data held by these operators is given by
+**Correctness**
+Let's show that any sets $U_q$ and $U_a$ satisfying the constraints in the [Consensus Layer Overview](../overview.md#consensus-layer), the data held by the operators $U_q \setminus U_a$ will constitute an entire blob. The amount of data held by these operators is given by
 
 $$
 \sum_{i \in U_q \setminus U_a} m_i C
@@ -79,7 +58,8 @@ $$
 \sum_{i \in U_q \setminus U_a} m_i C \ge  =\frac{B}{\gamma}\sum_{i \in U_q \setminus U_a}\frac{S_i}{\sum_j S_j} = \frac{B}{\gamma}\frac{\sum_{i \in U_q} S_i - \sum_{i \in U_a} S_i}{\sum_jS_j} \ge B \frac{\beta-\alpha}{\gamma} = B  \tag{2}
 $$
 
-Thus, the reconstruction requirement from the [Encoding](./encoding.md) module is satisfied. 
+Since the unique data held by these operators exceeds the size of a blob, the encoding module ensures that the original blob can be reconstructed from this data. 
+
 
 ## Validation Actions
 
@@ -91,7 +71,7 @@ When the DA node receives a `StoreChunks` request, it performs the following val
 - It uses the `ValidateChunkLength` to validate that the `ChunkLength` for the blob satisfies the above constraints. 
 - It uses `GetOperatorAssignment` to calculate the chunk indices for which it is responsible, and verifies that each of the chunks that it has received lies on the polynomial at these indices (see [Encoding validation actions](./encoding.md#validation-actions))
 
-This step ensures that each honest node has received the blobs for which it is accountable under the [Standard Assignment Coordinator](#standard-assignment-security-logic).
+This step ensures that each honest node has received the blobs for which it is accountable.
 
 Since the DA nodes will allow a range of `ChunkLength` values, as long as they satisfy the constraints of the protocol, it is necessary for there to be consensus on the `ChunkLength` that is in use for a particular blob and quorum. For this reason, the `ChunkLength` is included in the `BlobQuorumParam` which is hashed to create the merkle root contained in the `BatchHeaderHash` signed by the DA nodes. 
 

docs/spec/consensus/bridging.md

@@ -0,0 +1,32 @@
+## Signature verification and bridging
+
+![image](../../assets/bridging-module.png)
+
+### L1 Bridging
+
+Bridging a DA attestion for a specific blob requires the following stages:
+- *Bridging the batch attestation*. This involves checking the aggregate signature of the DA nodes for the batch, and tallying up the total amount of stake the signing nodes.
+- *Verifying the blob inclusion*. Each batch contains a the root of a a Merkle tree whose leaves correspond to the blob headers contained in the batch. To verify blob inclusion, the associate Merkle proof must be supplied and evaluated. Furthermore, the specific quorum threshold requirement for the blob must be checked against the total amount of signing stake for the batch. 
+
+For the first stage, EigenDA makes use of the EigenLayer's default utilities for managing operator state, verifying aggregate BLS signatures, and checking the total stake held by the signing operators.
+
+For the second stage, the EigenDA provides a utility contract with a `verifyBlob` method which rollups would typically integrate into their fraud proof pathway in the following manner: 
+1. The rollup sequencer posts all lookup data needed to verify a blob against a batch to the rollup inbox contract. 
+2. To initiate a fraud proof, the challenger must call the `verifyBlob` method with the supplied lookup data. If the blob does not verify correctly, the blob is considered invalid. 
+
+#### Reorg behavior (needs to be rewritten)
+
+One aspect of the chain behavior of which the attestation protocol must be aware is that of chain reorganization. The following requirements relate to chain reorganizations:
+1. Signed attestations should remain valid under reorgs so that a disperser never needs to resend the data and gather new signatures.
+2. If an attestation is reorged out, a disperser should always be able to simply resubmit it after a specific waiting period.
+3. Payloads constructed by a disperser and sent to DA nodes should never be rejected due to reorgs.
+
+These requirements result in the following design choices:
+- Chunk allotments should be based on registration state from a finalized block.
+- If an attestation is reorged out and if the transaction containing the header of a batch is not present within `BLOCK_STALE_MEASURE` blocks since `referenceBlockNumber` and the block that is `BLOCK_STALE_MEASURE` blocks since `referenceBlockNumber`  is finalized, then the disperser should again start a new dispersal with that blob of data. Otherwise, the disperser must not re-submit another transaction containing the header of a batch associated with the same blob of data.
+- Payment payloads sent to DA nodes should only take into account finalized attestations.
+
+The first and second decisions satisfy requirements 1 and 2. The three decisions together satisfy requirement 3.
+
+Whenever the `confirmBatch` method of the [ServiceMananger.sol](../contracts-service-manager.md) is called, the following checks are used to ensure that only finalized registration state is utilized:
+- Stake staleness check. The `referenceBlockNumber` is verified to be within `BLOCK_STALE_MEASURE` blocks before the confirmation block.This is to make sure that batches using outdated stakes are not confirmed. It is assured that stakes from within `BLOCK_STALE_MEASURE` blocks before confirmation are valid by delaying removal of stakes by `BLOCK_STALE_MEASURE + MAX_DURATION_BLOCKS`.