neutrino: add fast initial sync via header import#10552
neutrino: add fast initial sync via header import#10552Roasbeef wants to merge 6 commits intolightningnetwork:masterfrom
Conversation
Summary of ChangesHello @Roasbeef, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a significant performance improvement for LND nodes utilizing the Neutrino backend. By integrating a 'fast initial sync' mechanism, users can now bootstrap their nodes using pre-existing block and filter headers, drastically cutting down the time required to get a node up and running. This enhancement is designed to make LND more accessible and efficient for new users or those re-syncing, without compromising on security through robust validation and a P2P fallback. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
Update the neutrino dependency to a version that includes the chainimport package for fast initial header sync, and the ResetHeaderState fix that allows P2P sync to continue after import. The updated neutrino dependency changes the ChainService.Start method signature to accept a context.Context parameter. Update all call sites to pass context.TODO() to maintain existing behavior.
Add BlockHeadersSource and FilterHeadersSource fields to the Neutrino config struct. These accept either local file paths or HTTP(S) URLs pointing to pre-built header files for fast initial sync. Add a Validate method that ensures both sources are specified together or both are empty.
Pass the configured BlockHeadersSource and FilterHeadersSource into neutrino's HeadersImportConfig when initializing the neutrino backend. Set blockchain.BFFastAdd validation flags for regtest and simnet to skip contextual timestamp checks on rapidly-mined blocks. Call Validate on the neutrino config before proceeding to catch misconfiguration early.
Add commented examples for the new neutrino.blockheaderssource and neutrino.filterheaderssource options, showing both URL-based import from block-dn.org and local file path usage.
Add an integration test that verifies neutrino header import from local files. The test mines blocks, starts a reference node to generate header files via normal P2P sync, copies those files with import metadata, then starts a new node configured to import headers from the prepared files. The test verifies the import node syncs to the chain tip and can continue syncing additional blocks mined after import via P2P, exercising the hybrid import-then-P2P sync path.
Document the neutrino fast sync feature that allows importing block and filter headers from local files or HTTP URLs on startup. Cover configuration for mainnet (block-dn.org), testnet3, testnet4, signet, file format details, security considerations, and troubleshooting.
Roasbeef
force-pushed
the
neutrino-fast-sync
branch
from
f3b044e to
4c6e587
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a valuable feature for fast initial sync in neutrino mode by importing pre-built block and filter headers. The implementation is well-rounded, including new configuration options, necessary dependency updates, and excellent documentation. The addition of a comprehensive integration test is particularly noteworthy as it ensures the end-to-end flow is working correctly. The code is clean and follows the project's style. My only suggestion for future improvement is to address the use of context.TODO() which was introduced to adapt to dependency changes, to ensure robust cancellation and timeout handling throughout the application.
| // Connect to bitcoind, and register for notifications on connected, | ||
| // and disconnected blocks. | ||
| if err := b.chainConn.Start(); err != nil { | ||
| if err := b.chainConn.Start(context.TODO()); err != nil { |
There was a problem hiding this comment.
Using context.TODO() is a pragmatic way to adapt to the updated dependency interface without a major refactoring. However, it introduces some technical debt. For improved robustness, especially during shutdown, it would be beneficial to propagate a proper context with cancellation capabilities from the callers. I recommend creating a follow-up issue to address this throughout the codebase where context.TODO() has been introduced.
🔴 PR Severity: CRITICAL
🔴 Critical (1 file)
🟠 High (4 files)
🟡 Medium (3 files)
🟢 Low (9 files)
AnalysisThis PR is classified as CRITICAL because it modifies The PR introduces neutrino's Key concerns for reviewers:
The integration test coverage is thorough and the documentation is comprehensive, which is positive. However, given this touches the wallet's chain backend initialization path, it requires expert review. To override, add a |
2 participants
This PR wires neutrino's
chainimportfeature into lnd, enabling fast initial sync by importing pre-built block and filter headers from local files or HTTP URLs before falling back to P2P sync for the remainder of the chain. On mainnet, this can reduce initial sync from hours to minutes by downloading headers from a trusted source like block-dn.org instead of fetching them one-by-one from peers.Two new config options are added under the
[neutrino]section:neutrino.blockheaderssourceandneutrino.filterheaderssource. Both accept either a local file path or an HTTP(S) URL. When configured, neutrino imports headers from these sources on startup, validates proof-of-work, and then continues syncing any remaining blocks via its normal P2P protocol. Both sources must be specified together or not at all.The neutrino dependency is updated to include the
chainimportpackage (PRs #317, #320, #324) as well as a bug fix for the block manager's internal state after import (lightninglabs/neutrino#335). Without this fix, the block manager would retain stale genesis-based locators after import and fail to sync additional blocks via P2P. TheStart()method signature change to acceptcontext.Contextis propagated to all call sites.An integration test exercises the full end-to-end flow: mining blocks, syncing a reference node to generate header files, copying those files with import metadata, starting a fresh node with the import config, and verifying it syncs to the chain tip and can continue syncing additional blocks mined after the import snapshot.
Depends on lightninglabs/neutrino#335.