v2.7.6

• Added bucketed prometheus metrics head_block_relative_time_sum to help investigate latency and pipeline performance:
  • "app=firehose_output" and "app=substreams_output" that shows latency between outputing live blocks and their blocktime.
  • "app=relayer" for latency at relayer's input
  • "app=reader-node" for latency at reader's input
• Bump base64 library to use a much faster one in reader
• dstore: bumped google storage lib to v1.59.1 to fix a bug in their multi-range downloader, in case it affects us

Substreams

• Fixed underflow in 'FailedPrecondition desc = request needs to process a total of x blocks' error when running from 'substreams run' with a start-block in the future.

v1.12.6

• Added bucketed prometheus metrics head_block_relative_time_sum to help investigate latency and pipeline performance:
  • "app=firehose_output" and "app=substreams_output" that shows latency between outputing live blocks and their blocktime.
  • "app=relayer" for latency at relayer's input
  • "app=reader-node" for latency at reader's input
• Bump base64 library to use a much faster one in reader
• dstore: bumped google storage lib to v1.59.1 to fix a bug in their multi-range downloader, in case it affects us

Substreams

• Fixed underflow in 'FailedPrecondition desc = request needs to process a total of x blocks' error when running from 'substreams run' with a start-block in the future.

v2.7.5

Substreams fixes

• Fix issue where "live backfiller" would not create segments after reconnecting with a cursor starting from a previous quicksave, causing delays in future reconnection
• Prevent "panic" when log messages are too large: instead, they will be truncated with a 'some logs were truncated' message.
• Raise max individual log message size from 128k to 512k
• Raise max log message size for a full block from 128k to 5MiB
• Reduce log level from Warn to Debug when we fail to get or set the store size (for backends that don't support it)

AWS Store

• Migrated from AWS SDK for Go v1 to v2 (github.com/aws/aws-sdk-go → github.com/aws/aws-sdk-go-v2)

Azure store

• Added support for "workload identity credentials" in Azure. Order of preference is:
  • If AZURE_STORAGE_KEY is set, use shared key credential (previous behavior)
    • Otherwise, use DefaultAzureCredential which supports:
      • Managed Identity (for Azure resources)
      • Service Principal (via AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID)
      • Azure CLI credentials
      • Visual Studio Code credentials

v1.12.5

Substreams fixes

• Fix issue where "live backfiller" would not create segments after reconnecting with a cursor starting from a previous quicksave, causing delays in future reconnection
• Prevent "panic" when log messages are too large: instead, they will be truncated with a 'some logs were truncated' message.
• Raise max individual log message size from 128k to 512k
• Raise max log message size for a full block from 128k to 5MiB
• Reduce log level from Warn to Debug when we fail to get or set the store size (for backends that don't support it)

AWS Store

• Migrated from AWS SDK for Go v1 to v2 (github.com/aws/aws-sdk-go → github.com/aws/aws-sdk-go-v2)

Azure store

• Added support for "workload identity credentials" in Azure. Order of preference is:
  • If AZURE_STORAGE_KEY is set, use shared key credential (previous behavior)
    • Otherwise, use DefaultAzureCredential which supports:
      • Managed Identity (for Azure resources)
      • Service Principal (via AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID)
      • Azure CLI credentials
      • Visual Studio Code credentials

v2.7.4

• Added flag --include-partial-blocks on tools firehose-client
• Substreams: fixed issue where "live backfiller" would not create segments after reconnecting with a cursor starting from a previous quicksave, causing delays in future reconnection

v1.12.4

• Added flag --include-partial-blocks on tools firehose-client
• Substreams: fixed issue where "live backfiller" would not create segments after reconnecting with a cursor starting from a previous quicksave, causing delays in future reconnection

v2.7.3

Substreams

• Fixed substreams regression in v1.12.2 where some jobs would not get scheduled correctly, resulting in failure with the mssage get size of store "...": opening file: not found.

v1.12.3

Substreams

• Fixed substreams regression in v1.12.2 where some jobs would not get scheduled correctly, resulting in failure with the mssage get size of store "...": opening file: not found.

v2.7.2

Substreams

• Fix egress bytes calculation when running in noop or dev mode with specified output debug modules.
• Add support to foundation store v2 protocol.
• Reduced memory usage when loading large stores
• Added opt-in memory limits related to loading FullKV stores, gated by environment variables:
  • "SUBSTREAMS_STORE_SIZE_LIMIT_PER_REQUEST" (default allows 5GiB: 5368709120): limit size of all loaded stores for a single request, in bytes. Set to a numeric value in bytes.
  • "SUBSTREAMS_ENFORCE_STORE_SIZE_LIMIT_PER_REQUEST" (default false): if set to true, enforce the limit above instead of just logging a warning
  • "SUBSTREAMS_TOTAL_STORE_SIZE_LIMIT_PERCENT" (default: 75): limit the size in-memory of all loaded stores concurrently on the instance, in percentage of usable memory (cgroup or system total -- regardless of free or available)
  • "SUBSTREAMS_ENFORCE_TOTAL_STORE_SIZE_LIMIT" (default: false): if set to true, enforce the limit above instead of just logging a warning
• Fixed an edge case where substreams with modules depending on stores that start on the future would fail and incorrectly report an error about "tier2 version being incompatible"

Firehose

• Added firehose_session_denied_counter with reason label, increment each time a session is refused with the reason why it was refused.

v1.12.2

Substreams

• Fix egress bytes calculation when running in noop or dev mode with specified output debug modules.
• Add support to foundation store v2 protocol.
• Reduced memory usage when loading large stores
• Added opt-in memory limits related to loading FullKV stores, gated by environment variables:
  • "SUBSTREAMS_STORE_SIZE_LIMIT_PER_REQUEST" (default allows 5GiB: 5368709120): limit size of all loaded stores for a single request, in bytes. Set to a numeric value in bytes.
  • "SUBSTREAMS_ENFORCE_STORE_SIZE_LIMIT_PER_REQUEST" (default false): if set to true, enforce the limit above instead of just logging a warning
  • "SUBSTREAMS_TOTAL_STORE_SIZE_LIMIT_PERCENT" (default: 75): limit the size in-memory of all loaded stores concurrently on the instance, in percentage of usable memory (cgroup or system total -- regardless of free or available)
  • "SUBSTREAMS_ENFORCE_TOTAL_STORE_SIZE_LIMIT" (default: false): if set to true, enforce the limit above instead of just logging a warning
• Fixed an edge case where substreams with modules depending on stores that start on the future would fail and incorrectly report an error about "tier2 version being incompatible"

Firehose

• Added firehose_session_denied_counter with reason label, increment each time a session is refused with the reason why it was refused.