Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: verify the range of the namespace IDs of the children in the HashNode #102

Merged
merged 13 commits into from
Feb 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 27 additions & 14 deletions hasher.go
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,9 @@ func (n *Hasher) Sum([]byte) []byte {
case NodePrefix:
flagLen := int(n.NamespaceLen) * 2
sha256Len := n.baseHasher.Size()
return n.HashNode(n.data[:flagLen+sha256Len], n.data[flagLen+sha256Len:])
leftChild := n.data[:flagLen+sha256Len]
rightChild := n.data[flagLen+sha256Len:]
return n.HashNode(leftChild, rightChild)
default:
panic("nmt node type wasn't set")
}
Expand Down Expand Up @@ -135,19 +137,23 @@ func (n *Hasher) HashLeaf(leaf []byte) []byte {

// HashNode calculates a namespaced hash of a node using the supplied left and
// right children. The input values, "left" and "right," are namespaced hash
// values with the format "minNID || maxNID || hash." By default, the normal
// namespace hash calculation is followed, which is "res = min(left.minNID,
// right.minNID) || max(left.maxNID, right.maxNID) || H(NodePrefix, left,
// right)". "res" refers to the return value of the HashNode. However, if the
// "ignoreMaxNs" property of the Hasher is set to true, the calculation of the
// namespace ID range of the node slightly changes. In this case, when setting
// the upper range, the maximum possible namespace ID (i.e.,
// 2^NamespaceIDSize-1) should be ignored if possible. This is achieved by
// taking the maximum value among the namespace IDs available in the range of
// its left and right children (i.e., max(left.minNID, left.maxNID ,
// right.minNID, right.maxNID)), which is not equal to the maximum possible
// namespace ID value. If such a namespace ID does not exist, the maximum NID is
// calculated as normal, i.e., "res.maxNID = max(left.maxNID , right.maxNID).
// values with the format "minNID || maxNID || hash." The HashNode function may
// panic if the inputs provided are invalid, i.e., when left and right are not
// in the namespaced hash format or when left.maxNID is greater than
// right.minNID. To prevent panicking, it is advisable to check these criteria
// before calling the HashNode function. By default, the normal namespace hash
// calculation is followed, which is "res = min(left.minNID, right.minNID) ||
// max(left.maxNID, right.maxNID) || H(NodePrefix, left, right)". "res" refers
// to the return value of the HashNode. However, if the "ignoreMaxNs" property
// of the Hasher is set to true, the calculation of the namespace ID range of
// the node slightly changes. In this case, when setting the upper range, the
// maximum possible namespace ID (i.e., 2^NamespaceIDSize-1) should be ignored
// if possible. This is achieved by taking the maximum value among the namespace
// IDs available in the range of its left and right children (i.e.,
// max(left.minNID, left.maxNID , right.minNID, right.maxNID)), which is not
// equal to the maximum possible namespace ID value. If such a namespace ID does
// not exist, the maximum NID is calculated as normal, i.e., "res.maxNID =
// max(left.maxNID , right.maxNID).
func (n *Hasher) HashNode(left, right []byte) []byte {
h := n.baseHasher
h.Reset()
Expand All @@ -158,6 +164,13 @@ func (n *Hasher) HashNode(left, right []byte) []byte {
leftMinNs, leftMaxNs := left[:n.NamespaceLen], left[n.NamespaceLen:flagLen]
rightMinNs, rightMaxNs := right[:n.NamespaceLen], right[n.NamespaceLen:flagLen]

// check the namespace range of the left and right children
rightMinNID := namespace.ID(rightMinNs)
leftMaxNID := namespace.ID(leftMaxNs)
if rightMinNID.Less(leftMaxNID) {
panic("nodes are out of order: the maximum namespace of the left child is greater than the min namespace of the right child")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would panicking here mean that someone could send a light node an invalid proof or sample and their node would crash?

Copy link
Contributor Author

@staheri14 staheri14 Feb 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you mentioned, the HashNode method may only fail in the event of an attack on the system. However, it is important to note that the method was previously susceptible to panic, even prior to this pull request. For example, panic can occur while extracting the namespace identifier of the left or right children if they are not namespaced (i.e., if their length is less than NamespaceLen). It is not possible to return an error from this method without violating the interfaces it is intended to fulfill, namely NodeHasher and hash.
At this time, it may be better to allow the light client to crash rather than enter an invalid state.

I will investigate the implementation of proper error handling in the HashNode method, including handling other potential errors beyond just the namespace ID range, in a follow-up pull request. As this has a broad scope, it requires careful attention and consideration during implementation.
Tracking issue #109

Copy link
Member

@liamsi liamsi Feb 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I think when this was implemented, we might have panic'ed bc it was supposed to be impossible to construct an invalid tree. But it is good thinking that someone might do so to crash light nodes and I do think we should error instead.

edit: I see. The Hash interface was another reason. Either we make it impossible client side that this panics (e.g. by doing some verification before calling this; then this should be very explicitly documented) or we have to break with the iface.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the sake of getting this PR merged: can we add this property/panic to the godoc? So the caller has a hint that verification has to happen before to not cause unexpected panics?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the sake of getting this PR merged: can we add this property/panic to the godoc? So the caller has a hint that verification has to happen before to not cause unexpected panics?

Sure! please check the following commit e7415b7

}

minNs := min(leftMinNs, rightMinNs)
var maxNs []byte
if n.ignoreMaxNs && n.precomputedMaxNs.Equal(leftMinNs) {
Expand Down
51 changes: 43 additions & 8 deletions hasher_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,14 +89,6 @@ func Test_namespacedTreeHasher_HashNode(t *testing.T) {
sum(crypto.SHA256, []byte{NodePrefix}, []byte{0, 0, 0, 0}, []byte{0, 0, 1, 1})...,
),
},
{
"leftmin==rightmin && leftmax>rightmax", 2,
children{[]byte{0, 0, 1, 1}, []byte{0, 0, 0, 1}},
append(
[]byte{0, 0, 1, 1},
sum(crypto.SHA256, []byte{NodePrefix}, []byte{0, 0, 1, 1}, []byte{0, 0, 0, 1})...,
Copy link
Contributor Author

@staheri14 staheri14 Feb 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test case is deleted as this represents an invalid state given the newly added logic i.e., the max namespace of the right child cannot be greater than the min namespace of the right child.

),
},
// XXX: can this happen in practice? or is this an invalid state?
{
"leftmin>rightmin && leftmax<rightmax", 2,
Expand Down Expand Up @@ -197,3 +189,46 @@ func TestNamespaceHasherSum(t *testing.T) {
})
}
}

func TestHashNode_ChildrenNamespaceRange(t *testing.T) {
type children struct {
l []byte // namespace hash of the left child with the format of MinNs||MaxNs||h
r []byte // namespace hash of the right child with the format of MinNs||MaxNs||h
}

tests := []struct {
name string
nidLen namespace.IDSize
children children
wantPanic bool // whether the test should panic or nor
}{
{
"left.maxNs>right.minNs", 2,
children{[]byte{0, 0, 1, 1}, []byte{0, 0, 1, 1}},
true, // this test case should panic since in an ordered NMT, left.maxNs cannot be greater than right.minNs
},
{
"left.maxNs=right.minNs", 2,
children{[]byte{0, 0, 1, 1}, []byte{1, 1, 2, 2}},
false,
},
{
"left.maxNs<right.minNs", 2,
children{[]byte{0, 0, 1, 1}, []byte{2, 2, 3, 3}},
false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
defer func() {
gotPanic := false
if r := recover(); r != nil { // here we check whether panic happened
gotPanic = true
}
assert.Equal(t, tt.wantPanic, gotPanic)
}()
n := NewNmtHasher(sha256.New(), tt.nidLen, false)
n.HashNode(tt.children.l, tt.children.r)
})
}
}