Modular overlay architecture for SPIRE integration

[saiakhil2012]

Overview

This PR introduces an overlay architecture for SPIRE integration with sovereign attestation capabilities, eliminating the need to maintain a full SPIRE fork. The architecture provides clear separation between upstream dependencies and custom extensions while adding support for hardware-based zero-knowledge proofs and mobile network operator endorsements.

Impact: Repository now tracks only 50 overlay files instead of maintaining a 17,315-file SPIRE fork, with added sovereign attestation features.

Architecture Overview

Implements a patch-based overlay architecture that applies custom modifications to upstream SPIRE at build time:

Components:

1. Proto Extensions: Custom protocol buffer definitions for sovereign attestation:

   • AegisSovereignAttestation - TPM-based attestation package
   • AegisAttestedClaims - Keylime claims with ZKP support
   • Geolocation - Mobile/GNSS sensor data
   • MNOEndorsement - Mobile network operator endorsements
   • All fields use optional for proto3 flexibility
   • Aegis prefix on parent messages for upstream merge safety

2. Core Patches: Attestation flow modifications for hardware-based trust and sovereign attestation

3. Plugin System:

   • Credential Composer (Unified Identity): Calls Keylime verifier, converts ZKP claims to proto types
   • Unified Identity Plugin: Embeds sovereign attestation data (geolocation, MNO endorsement, ZKP receipts) into X.509 SVIDs
   • TPM DevID node attestor
   • Policy engine integration

4. Automated Tooling: Clone → patch → build automation with version pinning

Structure:

spire-overlay/
├── proto-patches/ # Protocol buffer extensions
│ └── files/spire-api-sdk/spire/api/types/sovereignattestation.proto
├── core-patches/ # Patch files for SPIRE core
├── plugins/ # Custom plugin implementations
│ ├── server-credentialcomposer-unifiedidentity/ # Keylime integration
│ └── server-unifiedidentity/ # Claims embedding
├── packages/ # Shared utilities and caching
└── README.md # System documentation

scripts/
├── spire-build.sh # Automated build orchestration
├── spire-dev-setup.sh # Development environment generator
├── spire-dev-extract.sh # Patch extraction tool
└── spire-dev-cleanup.sh # Environment cleanup utility

Sovereign Attestation Features

ZKP (Zero-Knowledge Proof) Support

• Keylime Integration: Credential composer plugin fetches attested claims from Keylime verifier
• Field Conversion: Complete mapping from Keylime JSON to proto types:
  • Geolocation (11 fields: sensor ID, IMEI, IMSI, lat/long, accuracy, signature)
  • MNOEndorsement (verified flag, JSON payload, signature, key ID)
  • SovereigntyReceipt (ZKP proof string)
• SVID Embedding: Unified identity plugin embeds sovereign claims in X.509 extension JSON
• Data Flow: mobile-sensor → Keylime → SPIRE plugin → SVID certificate

Proto Naming Strategy

Parent messages use Aegis prefix to prevent upstream conflicts:

• message AegisSovereignAttestation ✅
• message AegisAttestedClaims ✅

Child messages without prefix (project-specific nested structures):

• message Geolocation ✅
• message MNOEndorsement ✅

Rationale: When SPIRE upstream adds similar top-level types, Aegis prefix prevents naming collisions during merge.

Development Workflow

On-demand development environment:

1. Production mode: 50 overlay files, minimal repo footprint
2. Development mode: Temporary fork with full IDE support (LSP, debugger, refactoring)
3. Patch extraction: Automated diff generation preserving only custom changes
4. Cleanup: One-command restoration to production state

Enables full IDE capabilities during development while maintaining clean repository state.

Testing & Validation

Hardware Testing:

• ✅ Validated on Linux with TPM 2.0 hardware
• ✅ End-to-end integration test (complete attestation flow)
• ✅ SVID generation and automatic renewal verified
• ✅ mTLS client authentication with hardware-backed credentials
• ✅ 370 log entries captured across all components

Test Coverage:

• Control plane services initialization
• Agent attestation with hardware TPM
• Certificate issuance and rotation
• Network connectivity under load
• Service resilience and recovery
• ZKP field propagation (infrastructure complete, integration in progress)

Performance & Maintainability

Repository Metrics:

• Files tracked: 17,315 → 50 (99.7% reduction)
• Git operations: ~50x faster (clone, status, diff)
• Code changes: -231K lines net (removed fork, added overlay + ZKP plugins)

Maintainability:

• Clear separation of concerns (custom vs upstream code)
• Version updates: single variable change
• Patch conflicts: immediately visible at build time
• Code review: only custom logic reviewed
• Upstream tracking: straightforward rebase to newer SPIRE versions
• Proto naming ready for upstream merge (Aegis prefix prevents conflicts)

Documentation

• docs/SPIRE_DEV_WORKFLOW.md (237 lines) - Developer workflow guide
• docs/TPM_TESTING_CHECKLIST.md (247 lines) - Hardware validation procedures
• spire-overlay/README.md (176 lines) - Architecture overview
• spire-overlay/proto-patches/README.md - Proto patch mechanism

Each includes examples, troubleshooting, and best practices.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• I have signed-off my commits (DCO)

Commit Structure

1. Add SPIRE overlay system - Proto definitions, patch infrastructure, build scripts, and core plugins
2. Add ZKP propagation in unified identity plugins - Keylime integration, sovereign claims conversion, SVID embedding
3. Add build workflow and comprehensive documentation - Automated tooling and developer guides
4. Remove SPIRE fork - Eliminate 17,315-file dependency from version control

---

This architecture reduces maintenance overhead, improves git performance, adds sovereign attestation capabilities (TPM + ZKP + MNO endorsements), and provides a sustainable approach to integrating custom functionality with upstream dependencies.