Skip to content

ddr memory requrement in long-context (8k or more) #17146

@sigpro

Description

@sigpro

In long-context cases (e.g., Qwen2.5-3B or Qwen3-1.7B with 8K prefill), how much DDR memory is typically needed to run on a Qualcomm NPU?

cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: qnnIssues related to Qualcomm's QNN delegate and code under backends/qualcomm/partner: qualcommFor backend delegation, kernels, demo, etc. from the 3rd-party partner, Qualcomm

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions