Arm backend: Add FP16 (pt.8) and BF16 support to operators #17370

martinlsm · 2026-02-11T13:51:14Z

Add FP16 support for operators:

add
amax
amin
permute
sum

Add BF16 support for operators:

amax
amin

cc @freddan80 @per @zingo @oscarandersson8218 @digantdesai

Add FP16 support for operators: - add - amax - amin - permute - sum Signed-off-by: Martin Lindström <Martin.Lindstroem@arm.com> Change-Id: Iacdf79adf8e4ce9d16dce91f590f49eef339323d

Signed-off-by: Martin Lindström <Martin.Lindstroem@arm.com> Change-Id: I42d15ff8c62463fe628bad103ef24b7a11b8b4a3

pytorch-bot · 2026-02-11T13:51:18Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17370

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Awaiting Approval, 10 New Failures, 15 Unrelated Failures

As of commit c3b4336 with merge base 9c74c32 ():

AWAITING APPROVAL - The following workflow needs approval before CI can run:

periodic (gh)

NEW FAILURES - The following jobs have failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t 0474088be889436e24f923547ef58c825d046e82b17b10600605e7a8eeca1e41 /exec failed with exit code 1
pull / test-samsung-quantmodels-linux / linux-job (gh)
RuntimeError: Command docker exec -t a499470c4c26d702197538dba19f684e6a6e52a95d88fc975ebaa6939ae0af4d /exec failed with exit code 1
trunk / test-qnn-model (fp32, conv_former) / linux-job (gh)
RuntimeError: Command docker exec -t 8ba92a67eb0034da6ca5124c5c70647d52028e0a4f861d2d167574b5aaf6dbfd /exec failed with exit code 137
trunk / test-qnn-model (fp32, dl3) / linux-job (gh)
RuntimeError: Command docker exec -t 21d467b933ca546b37d294aa583dd7c34380ffd30570a6bd5ec42e41839bce63 /exec failed with exit code 137
trunk / test-qnn-model (fp32, ic3) / linux-job (gh)
RuntimeError: Command docker exec -t 9bde76dd8a6098886ff0b0704561191e116766b2b9954923be371738c708d6e4 /exec failed with exit code 137
trunk / test-qnn-model (fp32, mb) / linux-job (gh)
RuntimeError: Command docker exec -t 7d911220fced27106e0e8c8b7afe32e82dbf9fa8877e35a80d231c6ae5003b3d /exec failed with exit code 137
trunk / test-qnn-model (fp32, mv2) / linux-job (gh)
RuntimeError: Command docker exec -t dbc9d171a3aa7760c09e00e1b4ae3ac215b3aeef5f5993cd56db044e6d6a0f0d /exec failed with exit code 137
trunk / test-qnn-model (fp32, vit) / linux-job (gh)
RuntimeError: Command docker exec -t f9c48d1e8d0486541e8dd3b694bc3b0a3d330e06fe6162e6dc89987a3bfc2f91 /exec failed with exit code 137
trunk / test-qnn-optimum-model (fp32, dit) / linux-job (gh)
RuntimeError: Command docker exec -t ece4efbf711533b5da72775d86a796a5aa3e9898d8239ca892df98781950836b /exec failed with exit code 137
trunk / test-qnn-optimum-model (fp32, swin) / linux-job (gh)
RuntimeError: Command docker exec -t 218a93e3d2f8da762872f9ce45c8e0917d52cbc3e59934623a65911bed2538b9 /exec failed with exit code 137

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh) (matched linux rule in flaky-rules.json)
The runner has received a shutdown signal. This can happen when the runner service is stopped, or a manually started runner is canceled.
trunk / test-qnn-optimum-model (fp32, efficientnet) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
trunk / test-qnn-optimum-model (fp32, focalnet) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
trunk / test-qnn-optimum-model (fp32, mobilevit_v1) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
trunk / test-qnn-optimum-model (fp32, roberta) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

martinlsm · 2026-02-11T13:51:26Z

@pytorchbot label ciflow/trunk

martinlsm · 2026-02-11T13:51:36Z

@pytorchbot label "partner: arm"

martinlsm · 2026-02-11T13:51:50Z

@pytorchbot label "release notes: arm"

Copilot

Pull request overview

Adds FP16 (and for a subset, BF16) support coverage for the Arm backend by expanding operator dtype validation and extending the existing operator test suites.

Changes:

Allow FP16 (and BF16 where applicable) in Arm TOSA lowering dtype validation for add, sum, permute, amax, and amin.
Extend Arm backend operator tests to exercise FP16 (and BF16 for amax/amin) paths across TOSA FP and (where relevant) VGF pipelines.
Refactor some test parametrization/xfail mappings to separate FP vs INT expectations.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
backends/arm/test/ops/test_to_copy.py	Splits redundant-cast xfail mappings for FP vs INT pipelines.
backends/arm/test/ops/test_sum.py	Adds FP16 test coverage for `sum` (dim_IntList and default variants).
backends/arm/test/ops/test_permute.py	Adds FP16 test coverage for `permute` and simplifies test suite dict composition.
backends/arm/test/ops/test_amin.py	Adds FP16/BF16 test cases and uses explicit op-name constants for pipelines.
backends/arm/test/ops/test_amax.py	Adds FP16/BF16 test cases and uses explicit op-name constants for pipelines.
backends/arm/test/ops/test_add.py	Adds FP16/BF16 test cases for `add` variants and consolidates FP tests.
backends/arm/operators/op_sum.py	Allows FP16 dtype in `sum` dtype validation.
backends/arm/operators/op_permute.py	Allows FP16 dtype in `permute` dtype validation.
backends/arm/operators/op_amin.py	Allows FP16 and BF16 dtypes in `amin` dtype validation.
backends/arm/operators/op_amax.py	Allows FP16 and BF16 dtypes in `amax` dtype validation.
backends/arm/operators/op_add.py	Allows FP16 dtype in `add` dtype validation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-11T13:57:32Z

backends/arm/test/ops/test_amin.py

+amin_exir_op = "executorch_exir_dialects_edge__ops_aten_amin_default"
+
+min_aten_op = "torch.ops.aten.min"
+min_exir_op = "executorch_exir_dialects_edge__ops_aten_min_default"


amin_exir_op and min_exir_op are defined but never referenced in this file. Consider deleting them or using them in the relevant pipelines / not-delegated checks to avoid unused-variable lint issues and keep the test definitions focused.

Suggested change

amin_exir_op = "executorch_exir_dialects_edge__ops_aten_amin_default"

min_aten_op = "torch.ops.aten.min"

min_exir_op = "executorch_exir_dialects_edge__ops_aten_min_default"

min_aten_op = "torch.ops.aten.min"

Copilot · 2026-02-11T13:57:33Z

backends/arm/test/ops/test_add.py

+
+    test_data_fp16 = {
+        "4d_big_small_fp16": lambda: (
+            (10e10) * torch.randn(1, 10, 20, 30, dtype=torch.float16),


test_data_fp16 scales random FP16 values by 10e10 (~1e11), which will overflow FP16 (max ~6.5e4) and likely produce inf/-inf. This makes the test less meaningful and can hide numerical issues; consider using a scale within FP16 range (or generate values already bounded) so the test exercises real finite FP16 behavior.

Suggested change

(10e10) * torch.randn(1, 10, 20, 30, dtype=torch.float16),

(1e4) * torch.randn(1, 10, 20, 30, dtype=torch.float16),

Copilot · 2026-02-11T13:57:33Z

backends/arm/test/ops/test_add.py

+    test_data_fp16: list[input_t2] = {
+        "4d_randn_diff_rank_fp16": lambda: (
+            torch.randn(1, 1, 4, 4, dtype=torch.float16),
+            torch.randn(4, 1, dtype=torch.float16),
+        ),
+    }


The type annotation for test_data_fp16 is list[input_t2], but the assigned value is a dict keyed by test name. This mismatch can confuse readers and breaks static type checking; annotate it as a Dict[str, Callable[[], input_t2]] (or similar) to match usage.

Copilot · 2026-02-11T13:57:33Z

backends/arm/test/ops/test_add.py

+    test_data_bf16: list[input_t2] = {
+        "4d_randn_diff_rank_bf16": lambda: (
+            torch.randn(1, 1, 4, 4, dtype=torch.bfloat16),
+            torch.randn(4, 1, dtype=torch.bfloat16),
+        ),
+    }


The type annotation for test_data_bf16 is list[input_t2], but the assigned value is a dict keyed by test name. Update the annotation to a Dict[...] (matching how it’s used with common.parametrize and dict union) to avoid type confusion.

Copilot · 2026-02-11T13:57:34Z

backends/arm/test/ops/test_amax.py

+amax_exir_op = "executorch_exir_dialects_edge__ops_aten_amax_default"
+
+max_aten_op = "torch.ops.aten.max"
+max_exir_op = "executorch_exir_dialects_edge__ops_aten_max_default"


amax_exir_op and max_exir_op are defined but never used in this test module. If they’re not needed, remove them; if they are, prefer using them where you currently inline the EXIR op strings to keep the file consistent and avoid unused-symbol lint failures.

Suggested change

amax_exir_op = "executorch_exir_dialects_edge__ops_aten_amax_default"

max_aten_op = "torch.ops.aten.max"

max_exir_op = "executorch_exir_dialects_edge__ops_aten_max_default"

max_aten_op = "torch.ops.aten.max"

Martin Lindström added 2 commits February 11, 2026 14:42

Arm backend: Add FP16 support to operators pt.8

a25aef4

Add FP16 support for operators: - add - amax - amin - permute - sum Signed-off-by: Martin Lindström <Martin.Lindstroem@arm.com> Change-Id: Iacdf79adf8e4ce9d16dce91f590f49eef339323d

Arm backend: Add BF16 support to amin and amax

c3b4336

Signed-off-by: Martin Lindström <Martin.Lindstroem@arm.com> Change-Id: I42d15ff8c62463fe628bad103ef24b7a11b8b4a3

Copilot AI review requested due to automatic review settings February 11, 2026 13:51

martinlsm requested a review from digantdesai as a code owner February 11, 2026 13:51

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 11, 2026

pytorch-bot bot added the ciflow/trunk label Feb 11, 2026

pytorch-bot bot added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Feb 11, 2026

Copilot started reviewing on behalf of martinlsm February 11, 2026 13:51 View session

pytorch-bot bot added the release notes: arm Changes to the ARM backend delegate label Feb 11, 2026

Copilot AI reviewed Feb 11, 2026

View reviewed changes

zingo approved these changes Feb 11, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Arm backend: Add FP16 (pt.8) and BF16 support to operators #17370

Arm backend: Add FP16 (pt.8) and BF16 support to operators #17370

martinlsm commented Feb 11, 2026 •

edited by pytorch-bot bot

Loading

Uh oh!

pytorch-bot bot commented Feb 11, 2026 •

edited

Loading

Uh oh!

martinlsm commented Feb 11, 2026

Uh oh!

martinlsm commented Feb 11, 2026

Uh oh!

martinlsm commented Feb 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Copilot AI Feb 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	(10e10) * torch.randn(1, 10, 20, 30, dtype=torch.float16),
	(1e4) * torch.randn(1, 10, 20, 30, dtype=torch.float16),

Arm backend: Add FP16 (pt.8) and BF16 support to operators #17370

Are you sure you want to change the base?

Arm backend: Add FP16 (pt.8) and BF16 support to operators #17370

Conversation

martinlsm commented Feb 11, 2026 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17370

❌ 1 Awaiting Approval, 10 New Failures, 15 Unrelated Failures

Uh oh!

martinlsm commented Feb 11, 2026

Uh oh!

martinlsm commented Feb 11, 2026

Uh oh!

martinlsm commented Feb 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

martinlsm commented Feb 11, 2026 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Feb 11, 2026 •

edited

Loading