Add new StellarValue type representing a skipped ledger

[bboston7]

We would like to improve SCP performance by permitting some validator voting prior to receiving full transaction sets. In early voting stages (all of nomination, and part of balloting), validators would assume that transaction sets they do not posses are valid. This will speed up SCP, as transaction set dissemination is costly, and validators can safely make some progress on consensus while waiting to receive transaction sets.

In order to make this change, we need a method for a quorum of validators to decide that a transaction set is unavailable, and to vote to skip the current ledger. We expect these skipped ledgers to be rare.

To enable ledger skipping, we propose the following XDR change to Stellar-ledger.x:

diff --git a/Stellar-ledger.x b/Stellar-ledger.x
index a17036b..6ae2a60 100644
--- a/Stellar-ledger.x
+++ b/Stellar-ledger.x
@@ -13,7 +13,8 @@ typedef opaque UpgradeType<128>;
 enum StellarValueType
 {
     STELLAR_VALUE_BASIC = 0,
-    STELLAR_VALUE_SIGNED = 1
+    STELLAR_VALUE_SIGNED = 1,
+    STELLAR_VALUE_SKIP = 2
 };

 struct LedgerCloseValueSignature
@@ -43,6 +44,12 @@ struct StellarValue
         void;
     case STELLAR_VALUE_SIGNED:
         LedgerCloseValueSignature lcValueSignature;
+    case STELLAR_VALUE_SKIP:
+        struct
+        {
+            Hash txSetHash;
+            LedgerCloseValueSignature lcValueSignature;
+        } originalValue;
     }
     ext;
 };

With this change, the full StellarValue struct becomes:

/* StellarValue is the value used by SCP to reach consensus on a given ledger
 */
struct StellarValue
{
    Hash txSetHash;      // transaction set to apply to previous ledger
    TimePoint closeTime; // network close time

    // upgrades to apply to the previous ledger (usually empty)
    // this is a vector of encoded 'LedgerUpgrade' so that nodes can drop
    // unknown steps during consensus if needed.
    // see notes below on 'LedgerUpgrade' for more detail
    // max size is dictated by number of upgrade types (+ room for future)
    UpgradeType upgrades<6>;

    // reserved for future use
    union switch (StellarValueType v)
    {
    case STELLAR_VALUE_BASIC:
        void;
    case STELLAR_VALUE_SIGNED:
        LedgerCloseValueSignature lcValueSignature;
    case STELLAR_VALUE_SKIP:
        struct
        {
            Hash txSetHash;
            LedgerCloseValueSignature lcValueSignature;
        } originalValue;
    }
    ext;
};

When the network votes to skip the ledger, validators will externalize a StellarValue with fields set as follows:

• txSetHash : Zeroed out
• closeTime : Set as usual
• upgrades : Set as usual
• ext : Set to STELLAR_VALUE_SKIP
  • originalValue : Details about the original StellarValue that a quorum of validators decided to skip.
    • originalValue.txSetHash : The hash of the transaction set that was insufficiently disseminated.
    • originalValue.lcValueSignature : The signature from the original StellarValue. Can be used to determine which validator proposed the original StellarValue.

Note that the exact details of this struct are subject to change until solidified in a CAP.

Downstream Impacts

If this change is adopted, downstream consumers of StellarValues may require modification. Specifically, those that reason about txSetHash will need to handle the new 0x0 value, and those that reason about ext will need to handle the new STELLAR_VALUE_SKIP extension.

To help determine if you depend on something that contains a StellarValue, transitive dependencies on StellarValue are as follows:

StellarValue
    │
    ├──► LedgerHeader.scpValue
    │    │
    │    └──► LedgerHeaderHistoryEntry.header
    │         │
    │         ├──► LedgerCloseMetaV0.ledgerHeader
    │         ├──► LedgerCloseMetaV1.ledgerHeader
    │         └──► LedgerCloseMetaV2.ledgerHeader
    │              │
    │              └──► LedgerCloseMeta (union)
    │                   │
    │                   └──► LedgerCloseMetaBatch.ledgerCloseMetas
    │
    └──► StoredDebugTransactionSet.scpValue

Downstream consumers of StellarValues should treat skipped ledgers as empty ledgers. Emitted meta will reflect this.