asset-hub-polkadot, asset-hub-kusama: regenerate weights for staking and election pallets#1042
asset-hub-polkadot, asset-hub-kusama: regenerate weights for staking and election pallets#1042sigurpol wants to merge 15 commits intopolkadot-fellows:mainfrom
Conversation
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async |
|
Sorry, only fellows can run commands. |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has started 🚀 See logs here |
|
Thanks @PolkadotDom 🚀 |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:No changes found. |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_staking_async" has started 🚀 See logs here |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:No changes found. |
|
Ok I am going to ask again in the fellowship runner room |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_staking_async |
|
Sorry, only fellows can run commands. |
It failed to compile the PAH runtime. |
Both missing solc and the issue we were returning OK even if /cmd failed have been fixed in #1043 so I am rebasing on top of that. |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has started 🚀 See logs here |
… --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async'
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block_signed pallet_election_provider_multi_block_verifier pallet_election_provider_multi_block_unsigned pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:No changes found. |
|
@bkchr , @Ank4n from https://github.com/polkadot-fellows/runtimes/actions/runs/20483026864/job/58862929149#step:12:112 I took the command from #974 (comment) where it worked just fine... (also it hasn't errored out but returned success 😥) Extended logs: -- listing pallets for benchmark for asset-hub-polkadot
Pallets in {'name': 'asset-hub-polkadot', 'package': 'asset-hub-polkadot-runtime', 'path': 'system-parachains/asset-hubs/asset-hub-polkadot', 'uri': 'wss://polkadot-asset-hub-rpc.polkadot.io:443', 'is_relay': False, 'blocktime': 6000, 'extra_args': '--disable-mbm-checks', 'try_runtime_args': '--overwrite-state-version 1', 'build_extra_features': 'polkadot-ahm', 'benchmarks_templates': {'pallet_xcm_benchmarks::generic': 'templates/xcm-bench-template.hbs', 'pallet_xcm_benchmarks::fungible': 'templates/xcm-bench-template.hbs'}}: ['pallet_asset_rate', 'snowbridge_pallet_system_frontend', 'pallet_nfts', 'pallet_utility', 'pallet_child_bounties', 'pallet_xcm', 'pallet_parameters', 'pallet_treasury', 'pallet_timestamp', 'cumulus_pallet_parachain_system', 'pallet_referenda', 'pallet_xcm_benchmarks::generic', 'frame_system_extensions', 'pallet_uniques', 'pallet_multisig', 'pallet_collator_selection', 'pallet_bags_list', 'pallet_scheduler', 'pallet_whitelist', 'pallet_election_provider_multi_block', 'pallet_assets', 'pallet_transaction_payment', 'pallet_balances', 'pallet_election_provider_multi_block::verifier', 'pallet_xcm_benchmarks::fungible', 'pallet_conviction_voting', 'pallet_staking_async', 'pallet_asset_conversion', 'pallet_xcm_bridge_hub_router', 'pallet_proxy', 'pallet_bounties', 'pallet_ah_ops', 'pallet_indices', 'pallet_ah_migrator', 'pallet_message_queue', 'polkadot_runtime_common::claims', 'pallet_preimage', 'pallet_election_provider_multi_block::unsigned', 'pallet_session', 'pallet_election_provider_multi_block::signed', 'cumulus_pallet_xcmp_queue', 'pallet_asset_conversion_tx_payment', 'frame_system', 'pallet_vesting']
Pallet: ['pallet_election_provider_multi_block', 'pallet_election_provider_multi_block_signed', 'pallet_election_provider_multi_block_verifier', 'pallet_election_provider_multi_block_unsigned', 'pallet_staking_async']
Filtered out runtimes & pallets: {}
No pallets [['pallet_election_provider_multi_block', 'pallet_election_provider_multi_block_signed', 'pallet_election_provider_multi_block_verifier', 'pallet_election_provider_multi_block_unsigned', 'pallet_staking_async']] found in ['asset-hub-polkadot'] |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has started 🚀 See logs here |
|
thanks @bkontur , I was DM Basti as we speak to tell him that with |
…ot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async'
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:
|
… --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async'
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async" has finished ✅ See logs here DetailsSubweight results:
|
|
The genesis config includes devStakers: Some((2_000, 25_000)) which pre-populates 27,000 stakers (2,000 validators + 25,000 nominators). Many pallet_staking_async benchmarks call clear_validators_and_nominators() in their setup phase, which:
Even though this happens in the "setup" phase (before timing), the cost appears to leak into the measured execution time and this is the bug! Evidence Affected Benchmarks All benchmarks calling clear_validators_and_nominators():
Benchmarks using create_validator_with_nominators helper are also affected (e.g., process_offence_queue). |
|
if we just want to unlock staking very quickly, the easiest seems just adding frame_benchmarking::benchmarking::commit_db();at the end of the infamous Tested and works just fine for e.g. force_apply_min_commission and process_offence_queue. I've created an issue in the SDK: paritytech/polkadot-sdk#10798 |
|
I have merged paritytech/polkadot-sdk#10802 on SDK which will solve the bulk operation leaking into benchmark result. This change is needed in order to be able to remeasure weights decently for staking. |
|
The real fix comes from SDK paritytech/polkadot-sdk#11069 - we need to backport to 2512 - publish a new frame-omni-bencher - update deps - re-run benchmarks |
|
@ggwpez can you kind run (separately) /cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_clientand /cmd bench --runtime asset-hub-kusama --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client |
|
/cmd bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client |
|
/cmd bench --runtime asset-hub-kusama --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client |
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client" has started 🚀 See logs here |
|
Command "bench --runtime asset-hub-kusama --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client" has started 🚀 See logs here |
…t --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client'
|
Command "bench --runtime asset-hub-polkadot --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client" has finished ✅ See logs here DetailsSubweight results:
|
…--pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client'
|
Command "bench --runtime asset-hub-kusama --pallet pallet_election_provider_multi_block pallet_election_provider_multi_block::signed pallet_election_provider_multi_block::unsigned pallet_election_provider_multi_block::verifier pallet_staking_async pallet_staking_async_rc_client" has finished ✅ See logs here DetailsSubweight results:
|
|
@ggwpez , @kianenigma new weights regenerated - and our usual suspects are back to microseconds range |
ggwpez
left a comment
There was a problem hiding this comment.
For posterity:
Details
| File | Extrinsic | Old | New | Change [%] |
|---|---|---|---|---|
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block.rs | manage_fallback | 736.90ms | 1.14s | +54.60 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | cancel_deferred_slash | 2.54ms | 3.27ms | +28.90 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block_unsigned.rs | submit_unsigned | 27.63ms | 34.33ms | +24.25 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block_verifier.rs | on_initialize_invalid_terminal | 37.54ms | 46.11ms | +22.82 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block_verifier.rs | on_initialize_valid_terminal | 35.78ms | 43.92ms | +22.73 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block_verifier.rs | on_initialize_valid_non_terminal | 5.57ms | 6.26ms | +12.38 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block_verifier.rs | on_initialize_invalid_non_terminal | 3.67ms | 4.06ms | +10.62 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | force_new_era_always | 116.54us | 125.59us | +7.77 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block_signed.rs | register_not_full | 544.74us | 579.80us | +6.44 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async_rc_client.rs | set_keys | 659.06us | 700.20us | +6.24 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | force_new_era | 116.45us | 122.98us | +5.61 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_snapshot_rest | 118.96ms | 125.13ms | +5.19 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block_signed.rs | submit_page | 748.27us | 786.16us | +5.06 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | set_controller | 436.20us | 414.06us | -5.08 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | prune_era_reward_points | 288.48us | 271.86us | -5.76 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | cancel_deferred_slash | 150.65us | 141.78us | -5.89 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | bond | 684.99us | 643.40us | -6.07 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | prune_era_validator_reward | 289.52us | 271.69us | -6.16 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | restore_ledger | 445.43us | 414.61us | -6.92 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_snapshot_msp | 64.44ms | 59.97ms | -6.94 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async_rc_client.rs | purge_keys | 650.43us | 605.14us | -6.96 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block_verifier.rs | on_initialize_valid_non_terminal | 4.34ms | 4.04ms | -7.09 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | update_payee | 232.14us | 215.25us | -7.28 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | force_new_era | 121.34us | 112.48us | -7.30 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | withdraw_unbonded_update | 630.41us | 582.51us | -7.60 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_snapshot_rest | 107.23ms | 98.67ms | -7.98 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | force_new_era_always | 122.72us | 112.70us | -8.17 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | migrate_currency | 498.96us | 452.35us | -9.34 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | set_payee | 200.20us | 181.24us | -9.47 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | force_no_eras | 124.73us | 112.58us | -9.74 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block_unsigned.rs | validate_unsigned | 913.53us | 820.76us | -10.15 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async_rc_client.rs | set_keys | 718.92us | 643.20us | -10.53 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | apply_slash | 373.43ms | 325.66ms | -12.79 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_signed | 108.48ms | 94.20ms | -13.17 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | rc_on_offence | 172.09ms | 148.36ms | -13.79 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | payout_stakers_alive_staked | 335.70ms | 278.70ms | -16.98 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | rc_on_offence | 195.81ms | 145.28ms | -25.81 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block.rs | manage_fallback | 457.99ms | 305.15ms | -33.37 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block.rs | admin_set | 1.55ms | 607.34us | -60.81 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_signed_validation | 771.41us | 223.09us | -71.08 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_unsigned | 1.12ms | 247.01us | -78.02 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | nominate | 9.64ms | 1.86ms | -80.67 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | rc_on_session_report | 11.34ms | 2.00ms | -82.33 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | withdraw_unbonded_kill | 10.99ms | 1.91ms | -82.60 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | force_unstake | 10.92ms | 1.89ms | -82.68 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | reap_stash | 11.04ms | 1.83ms | -83.43 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | chill | 6.22ms | 1.02ms | -83.56 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | bond_extra | 8.20ms | 1.29ms | -84.28 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | unbond | 8.44ms | 1.22ms | -85.56 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | rebond | 8.41ms | 1.14ms | -86.47 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | chill_other | 8.37ms | 1.12ms | -86.64 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | rc_on_session_report | 13.74ms | 1.73ms | -87.42 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | withdraw_unbonded_kill | 15.36ms | 1.85ms | -87.98 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | nominate | 14.44ms | 1.53ms | -89.37 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | force_unstake | 17.28ms | 1.83ms | -89.44 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | reap_stash | 16.96ms | 1.77ms | -89.58 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | unbond | 11.78ms | 1.16ms | -90.11 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block.rs | admin_set | 5.19ms | 501.35us | -90.34 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | bond_extra | 12.78ms | 1.22ms | -90.47 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_unsigned | 2.56ms | 236.93us | -90.74 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | chill | 11.24ms | 978.29us | -91.29 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | chill_other | 13.59ms | 1.07ms | -92.09 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | rebond | 13.79ms | 1.08ms | -92.15 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_election_provider_multi_block.rs | on_initialize_into_signed_validation | 2.90ms | 211.11us | -92.71 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | process_offence_queue | 12.84ms | 836.82us | -93.48 |
| system-parachains/asset-hubs/asset-hub-kusama/src/weights/pallet_staking_async.rs | force_apply_min_commission | 3.37ms | 202.03us | -94.00 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | process_offence_queue | 15.97ms | 766.43us | -95.20 |
| system-parachains/asset-hubs/asset-hub-polkadot/src/weights/pallet_staking_async.rs | force_apply_min_commission | 4.13ms | 186.31us | -95.49 |
Re-run staking and election benchmarks to include fix in frame-omni-bencher coming from SDK #11069.
Needed for Release 2.1.0.