Test failure: nodeos_read_terminate_at_block_if_lr_test

[linh2931]

https://github.com/AntelopeIO/spring/actions/runs/14412295392/job/40423204461?pr=1364

Traceback (most recent call last):
  File "/__w/spring/spring/build/tests/nodeos_read_terminate_at_block_test.py", line 331, in <module>
    success = executeTest(
  File "/__w/spring/spring/build/tests/nodeos_read_terminate_at_block_test.py", line 69, in executeTest
    checkStatus(testNode, testNodeArgs)
  File "/__w/spring/spring/build/tests/nodeos_read_terminate_at_block_test.py", line 121, in checkStatus
    assert head == termAtBlock, f"head {head} termAtBlock {termAtBlock}"
AssertionError: head 101 termAtBlock 100

[linh2931]

https://github.com/AntelopeIO/spring/actions/runs/14447751353/job/40513411112?pr=1333#step:4:993

[heifner]

Seems be be failing because terminate-at-block is 100.

info  2025-04-11T21:39:16.145 nodeos    controller.cpp:1449           transition_to_savann ] Transitioning to savanna, IF Genesis Block 99, IF Critical Block 99
debug 2025-04-11T21:39:16.145 nodeos    block_state.cpp:203           create_transition_bl ] Create transition block 100
info  2025-04-11T21:39:16.148 nodeos    controller.cpp:1494           operator()           ] Transition to instant finality happening after block 100, First IF Proper Block 101

votes are received after it hits 100 and starts to terminate.

debug 2025-04-11T21:39:17.980 net-0     fork_database.cpp:364         set_pending_savanna_ ] set fork db pending savanna lib 101: 000000657e86f7b8fc838a6e4ca3e6983cceeb4755cb2b0f1c72846ea7924460

On startup it processes that LIB and transitions moving past block 100

info  2025-04-11T21:40:32.795 nodeos    block_handle.cpp:52           read                 ] Loading chain_head block 100 000000642d7c22afe69b738fe95b8372c6c9c8802434a7ea783f9399a6e05831
info  2025-04-11T21:40:32.795 nodeos    controller.cpp:1682           should_replay_block_ ] no irreversible blocks need to be replayed from block log
debug 2025-04-11T21:40:32.795 nodeos    fork_database.cpp:364         set_pending_savanna_ ] set fork db pending savanna lib 99: 00000063a22063b1a351a0ad4aa85fb20e923520e0617f096c7bc989d5d227eb
debug 2025-04-11T21:40:32.795 nodeos    fork_database.cpp:364         set_pending_savanna_ ] set fork db pending savanna lib 101: 000000657e86f7b8fc838a6e4ca3e6983cceeb4755cb2b0f1c72846ea7924460
info  2025-04-11T21:40:32.795 nodeos    controller.cpp:1866           operator()           ] fork database size 3 head 103 : 000000676584015363fae73f6e653a161fe548f7395c79d45cc299d3cb9aae4a, root 100 : 000000642d7c22afe69b738fe95b8372c6c9c8802434a7ea783f9399a6e05831

I think the only reliable way to solve this problem is to either check should_terminate() in a number of additional places. For example the setting of pending lib and processing of votes OR purge forkdb of non-validated blocks on shutdown when terminate-at-block is hit. Seems like purging forkdb would be preferable to adding all these checks for the extremely rare case someone is using the terminate-at-block feature.

[linh2931]

This problem happens on sync_call branch regularly.
https://github.com/AntelopeIO/spring/actions/runs/14550908389/job/40822128688?pr=1378

[heifner]

Discussed.

Would like to update truncate-at-block to work outside of replay. If a user wishes to not have any blocks beyond truncate-at-block in the forkdb they can specify both terminate-at-block and truncate-at-block. Seems for simplicity if both specified then nodeos should exit with error if block numbers do not match. Also nodeos should exit with error if truncate-at-block or terminate-at-block is for a block that has already been processed so that terminate/truncate is not possible.

truncate-at-block will purge non-validated(applied) blocks from the forkdb on shutdown.

[heifner]

  --truncate-at-block arg (=0)          Stop hard replay / block log recovery
                                        at this block number (if non-zero). Can
                                        also be used with terminate-at-block to
                                        prune any received blocks from fork
                                        database on exit.
  --terminate-at-block arg (=0)         Stops the node after reaching the
                                        specified block number (if non-zero).
                                        Use RPC endpoint /v1/producer/pause_at_
                                        block to pause at a specific block
                                        instead. Combine with
                                        --truncate-at-block to prune blocks
                                        beyond the specified number from the
                                        fork database on exit.