fix(trie): subtrie root node too small to have hash

[mediocregopher]

Background

We encountered a PST bug in one of the trie witness tests given the
following scenario:

Leafs 0x000...01 and 0x000..02 updated
root() called
Panic due to Lower subtrie root node at path Nibbles(0x000000000000000000000000000000000000000000000000000000000000000) has no hash

In this scenario we have the following nodes:

0x (root): extension node with short key 0x000000000000000000000000000000000000000000000000000000000000000
0x0x000000000000000000000000000000000000000000000000000000000000000: branch
0x0000000000000000000000000000000000000000000000000000000000000001: leaf w/ value 0x1
0x0000000000000000000000000000000000000000000000000000000000000002: leaf w/ value 0x2

The branch node is interesting because it is simultaneously the root of its lower subtrie, and its RLP encoding is <32 bytes and is therefore inlined into the parent extension rather than being hashed.

Our implementation of update_upper_subtrie_hashes was incorrectly assuming that the root node of lower subtries would always have its hash field populated, but that field only gets populated if the node is actually hashed.

Primary Solution

The solution is to refactor our sparse trie nodes to cache their RlpNode, rather than their Hash. This way in cases where the node encodes to a length smaller than 32 bytes we still cache the RlpNode, and have that available at the lower subtrie root.

Secondary Changes

• Introduced a new enum SparseNodeState to track whether or not a trie node's cached data is valid. This is mostly for code clarity and to make reasoning about the logic easier.

• Tests have been updated to always pass 32 byte keys into update/remove_leaf. Many tests were previously malformed and would pass smaller paths into those methods, creating tries which are technically possible from PST's perspective, but not supported in reality.

[github-actions]

✅ Changelog found on PR.

Edit changelog