feat(rpc): add gettxoutsetinfo endpoint

docs/json_rpc/gettxoutsetinfo.md

@@ -0,0 +1,3 @@
+# `gettxoutsetinfo`
+
+See [Zaino's Unspent Hash set](./gettxoutsetinfo/canonical_utxo_set_snapshot_hash.md) for more information on how the UTXO set hash is computed.

docs/json_rpc/gettxoutsetinfo/canonical_utxo_set_snapshot_hash.md

@@ -0,0 +1,169 @@
+    Title: ZAINO-UTXOSET-01 Canonical UTXO Set Snapshot Hash (v1)
+    Owners: dorianvp <dorianvp@zingolabs.org>
+            Za Wil <zancas@zingolabs.org>
+    Status: Draft
+    Category: Lightclients
+    Created: 2025-10-16
+    License: MIT
+
+## Terminology
+
+- The key words **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are to be interpreted as described in BCP 14 [^BCP14] when, and only when, they appear in all capitals..
+- Integers are encoded **little-endian** unless otherwise stated.
+- “CompactSize” refers to the [Bitcoin Specified](https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer) [Zcash Implementation](https://docs.rs/zcash_encoding/0.3.0/zcash_encoding/struct.CompactSize.html) of variable-length integer format.
+- `BLAKE3` denotes the 32-byte output of the BLAKE3 hash function.
+- This specification defines **version 1** (“V1”) of the ZAINO UTXO snapshot.
+- **network**:
+  a blockchain instance identified by its genesis block and consensus parameters.
+
+## Abstract
+
+This document specifies a deterministic, versioned procedure to compute a 32-byte hash of a node’s UTXO set at a specified best block. The snapshot uses a canonical ordering and serialization and is hashed under a domain tag.
+
+Among other uses, the snapshot hash can be used to:
+
+- Verify that two nodes at the same best block have the same UTXO set across implementations and versions.
+- Pin failing test fixtures to a snapshot hash to reproduce issues.
+- Log periodic hashes to show continuity of state over time.
+
+The hash is _not_ input to consensus validation.
+
+## Motivation
+
+Different nodes (e.g., `zcashd`, Zebra, indexers) may expose distinct internals or storage layouts. Operators often need a cheap way to verify “we’re looking at the same unspent set” without transporting the entire set. A canonical, versioned snapshot hash solves this.
+
+## Domain Separation
+
+Implementations **MUST** domain-separate the hash with the ASCII header:
+
+```
+"ZAINO-UTXOSET-V1\0"
+```
+
+Any change to the encoding rules or semantics **MUST** bump the domain string (e.g., `…-V2\0`) and is out of scope of this document.
+
+## Inputs
+
+To compute the snapshot hash, the implementation needs:
+
+- `network`: ASCII string identifying the chain. Recommended values: `"mainnet"`, `"testnet"`, `"regtest"`.
+- `best_height`: the best chain height at the time of the snapshot (unsigned 32-bit).
+- `best_block`: the 32-byte block hash of the best chain tip, in the node’s _canonical internal byte order_.
+- `UTXO set`: a finite multimap keyed by outpoints `(txid, vout)` to outputs `(value_zat, scriptPubKey)`, where:
+
+  - `txid` is a 32-byte transaction hash (internal byte order).
+  - `vout` is a 32-bit output index (0-based).
+  - `value_zat` is a non-negative amount in zatoshis, range-checked to the node’s monetary bounds (e.g., `0 ≤ value_zat ≤ MAX_MONEY`).
+  - `scriptPubKey` is a byte string.
+
+Implementations **MUST** reject negative values or out-of-range amounts prior to hashing.
+
+## Canonical Ordering
+
+The snapshot **MUST** be ordered as follows, independent of the node’s in-memory layout:
+
+1. Sort by `txid` ascending, comparing the raw 32-byte values as unsigned bytes.
+2. For equal `txid`s, sort by `vout` ascending (unsigned 32-bit).
+
+This ordering **MUST** be used for serialization.
+
+## Serialization
+
+The byte stream fed to the hash is the concatenation of a **header** and **entries**:
+
+### Header
+
+- ASCII bytes: `"ZAINO-UTXOSET-V1\0"`
+- `network` as ASCII bytes, followed by a single NUL byte `0x00`.
+- `best_height` as `u32` little-endian.
+- `best_block` as 32 raw bytes.
+- `count_txouts` as `u64` little-endian, where `count_txouts` is the total number of serialized entries below.
+
+### Entries (one per outpoint in canonical order)
+
+For each `(txid, vout, value_zat, scriptPubKey)`:
+
+- `txid` as 32 raw bytes.
+- `vout` as `u32` little-endian.
+- `value_zat` as `u64` little-endian.
+- `script_len` as CompactSize (Bitcoin/Zcash varint) of `scriptPubKey.len()`.
+- `scriptPubKey` raw bytes.
+
+**Note:** No per-transaction terminators or grouping markers are used. Instead, the format commits to _outputs_, not _transactions_.
+
+### CompactSize ([reference](https://en.bitcoin.it/wiki/Protocol_documentation#Variable_length_integer))
+
+- If `n < 0xFD`: a single byte `n`.
+- Else if `n ≤ 0xFFFF`: `0xFD` followed by `n` as `u16` little-endian.
+- Else if `n ≤ 0xFFFF_FFFF`: `0xFE` followed by `n` as `u32` little-endian.
+- Else: `0xFF` followed by `n` as `u64` little-endian.
+
+## Hash Function
+
+- The implementation **MUST** stream the bytes above into a BLAKE3 hasher.
+- The 32-byte output of the hasher is the **snapshot hash**.
+
+## Pseudocode
+
+```text
+function UtxoSnapshotHashV1(network, best_height, best_block, utxos):
+    H ← blake3::Hasher()
+
+    // Header
+    H.update("ZAINO-UTXOSET-V1\0")
+    H.update(network)
+    H.update("\0")
+    H.update(le_u32(best_height))
+    H.update(best_block)         // 32 raw bytes, node’s canonical order
+    count ← number_of_outputs(utxos)
+    H.update(le_u64(count))
+
+    // Entries in canonical order
+    for (txid, vout, value, script) in sort_by_txid_then_vout(utxos):
+        assert 0 ≤ value ≤ MAX_MONEY
+        H.update(txid)                      // 32 raw bytes
+        H.update(le_u32(vout))
+        H.update(le_u64(value))             // zatoshis
+        H.update(CompactSize(script.len))
+        H.update(script)
+
+    return H.finalize() // 32-byte BLAKE3 digest
+```
+
+## Error Handling
+
+- If any `value_zat` is negative or exceeds `MAX_MONEY`, the snapshot procedure **MUST** fail and **MUST NOT** produce a hash.
+- If the UTXO set changes during iteration (non-atomic read), the implementation **SHOULD** retry using a stable view (e.g., read lock or height-pinned snapshot).
+
+## Security and Interop Considerations
+
+- This hash is **not a consensus commitment** and **MUST NOT** be used to validate blocks or transactions.
+- The domain string prevents cross-protocol collisions.
+- Including `network`, `best_height`, and `best_block` prevents accidental equality across different nodes or heights.
+- Because the order is fully specified, two independent implementations reading the _same_ set will produce the _same_ hash.
+
+## Rationale
+
+- **BLAKE3** is chosen for speed and strong modern security. SHA-256 would also work but is slower in large sets. The domain string ensures local uniqueness regardless of the hash function family.
+- Committing to _outputs_ rather than _transactions_ simplifies implementations that don’t have transaction-grouped storage.
+- CompactSize matches existing Bitcoin/Zcash encoding and avoids ambiguity.
+
+## Versioning
+
+- Any breaking change to the byte stream, input semantics, or ordering **MUST** bump the domain tag to `ZAINO-UTXOSET-V2\0` (or higher).
+- Implementations **SHOULD** publish the version alongside the hash in logs and APIs.
+
+## Test Guidance
+
+Implementations **SHOULD** include tests covering:
+
+1. **Determinism:** Shuffle input, and the hash remains constant.
+2. **Sensitivity:** Flip one bit in `value_zat` or `scriptPubKey`, and the hash changes.
+3. **Metadata:** Change `network` or `best_block`, and the hash changes.
+4. **Empty Set:** With `count_txouts = 0`, the hash is well-defined.
+5. **Large Scripts:** Scripts with CompactSize boundaries (252, 253, 2^16, 2^32).
+6. **Ordering:** Two entries with same `txid` different `vout` are ordered by `vout`.
+
+## References
+
+[^BCP14]: [Information on BCP 14 — "RFC 2119"](https://www.rfc-editor.org/info/bcp14)

integration-tests/tests/fetch_service.rs

@@ -1,4 +1,6 @@
-//! These tests compare the output of `FetchService` with the output of `JsonRpcConnector`.
+//! These tests compare the output of `FetchService` with the output of [`JsonRpSeeConnector`].
+//!
+//! Note that they both rely on the [`JsonRpSeeConnector`] to get the data.
 
 use futures::StreamExt as _;
 use zaino_common::network::ActivationHeights;
@@ -502,6 +504,53 @@ async fn fetch_service_get_address_tx_ids(validator: &ValidatorKind) {
     test_manager.close().await;
 }
 
+async fn fetch_service_get_txout_set_info() {
+    let (mut test_manager, _fetch_service, fetch_service_subscriber) =
+        create_test_manager_and_fetch_service(&ValidatorKind::Zcashd, None, true, true, true, true)
+            .await;
+
+    let mut clients = test_manager
+        .clients
+        .take()
+        .expect("Clients are not initialized");
+    clients.faucet.sync_and_await().await.unwrap();
+
+    let recipient_ua = clients.get_recipient_address("unified").await;
+    let _tx = zaino_testutils::from_inputs::quick_send(
+        &mut clients.faucet,
+        vec![(&recipient_ua, 250_000, None)],
+    )
+    .await
+    .unwrap();
+
+    test_manager.local_net.generate_blocks(1).await.unwrap();
+    tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;
+
+    let txout_set_info = fetch_service_subscriber.get_txout_set_info().await.unwrap();
+
+    let jsonrpc_client = JsonRpSeeConnector::new_with_basic_auth(
+        test_node_and_return_url(
+            test_manager.zebrad_rpc_listen_address,
+            false,
+            None,
+            Some("xxxxxx".to_string()),
+            Some("xxxxxx".to_string()),
+        )
+        .await
+        .unwrap(),
+        "xxxxxx".to_string(),
+        "xxxxxx".to_string(),
+    )
+    .unwrap();
+    let json_rpc_txout_set_info = jsonrpc_client.get_txout_set_info().await.unwrap();
+    dbg!(&json_rpc_txout_set_info);
+    dbg!(&txout_set_info);
+
+    assert_eq!(txout_set_info, json_rpc_txout_set_info);
+
+    test_manager.close().await;
+}
+
 async fn fetch_service_get_address_utxos(validator: &ValidatorKind) {
     let (mut test_manager, _fetch_service, fetch_service_subscriber) =
         create_test_manager_and_fetch_service(validator, None, true, true, true, true).await;
@@ -1528,6 +1577,11 @@ mod zcashd {
             fetch_service_get_address_tx_ids(&ValidatorKind::Zcashd).await;
         }
 
+        #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+        pub(crate) async fn txout_set_info() {
+            fetch_service_get_txout_set_info().await;
+        }
+
         #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
         pub(crate) async fn address_utxos() {
             fetch_service_get_address_utxos(&ValidatorKind::Zcashd).await;

integration-tests/tests/json_server.rs

@@ -1,4 +1,4 @@
-//! Tests that compare the output of both `zcashd` and `zainod` through `FetchService`.
+//! Tests that compare the output of both `zcashd` and `zainod` through [`FetchService`].
 
 use zaino_common::network::ActivationHeights;
 use zaino_common::{DatabaseConfig, ServiceConfig, StorageConfig};
@@ -726,6 +726,50 @@ mod zcashd {
             test_manager.close().await;
         }
 
+        #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+        async fn get_txout_set_info() {
+            let (
+                mut test_manager,
+                _zcashd_service,
+                zcashd_subscriber,
+                _zaino_service,
+                zaino_subscriber,
+            ) = create_test_manager_and_fetch_services(false, true).await;
+
+            let mut clients = test_manager
+                .clients
+                .take()
+                .expect("Clients are not initialized");
+
+            test_manager.local_net.generate_blocks(1).await.unwrap();
+            tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;
+
+            clients.faucet.sync_and_await().await.unwrap();
+
+            let recipient_ua = &clients.get_recipient_address("unified").await;
+            let recipient_taddr = &clients.get_recipient_address("transparent").await;
+            from_inputs::quick_send(&mut clients.faucet, vec![(recipient_taddr, 250_000, None)])
+                .await
+                .unwrap();
+            from_inputs::quick_send(&mut clients.faucet, vec![(recipient_ua, 250_000, None)])
+                .await
+                .unwrap();
+
+            tokio::time::sleep(tokio::time::Duration::from_secs(1)).await;
+
+            let zcashd_subscriber_txout_set_info =
+                zcashd_subscriber.get_txout_set_info().await.unwrap();
+            let zaino_subscriber_txout_set_info =
+                zaino_subscriber.get_txout_set_info().await.unwrap();
+
+            assert_eq!(
+                zcashd_subscriber_txout_set_info,
+                zaino_subscriber_txout_set_info
+            );
+
+            test_manager.close().await;
+        }
+
         #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
         async fn get_peer_info() {
             let (