|
| 1 | +--- |
| 2 | +name: cortex-m |
| 3 | +description: Build, test, or develop the Cortex-M (CMSIS-NN) backend. Use when working on backends/cortex_m/, running Cortex-M tests, or exporting models for Cortex-M targets. |
| 4 | +--- |
| 5 | + |
| 6 | +# Cortex-M (CMSIS-NN) Backend |
| 7 | + |
| 8 | +## Architecture |
| 9 | + |
| 10 | +Not a delegate backend — no partitioner. Custom ops and graph passes replace ATen quantized ops with CMSIS-NN equivalents at the graph level. |
| 11 | + |
| 12 | +## Pipeline |
| 13 | + |
| 14 | +Uses standard PT2E quantization (`prepare_pt2e` / `convert_pt2e`), then `CortexMPassManager` rewrites quantized ops to `cortex_m::` equivalents. |
| 15 | + |
| 16 | +```python |
| 17 | +from executorch.backends.cortex_m.quantizer.quantizer import CortexMQuantizer |
| 18 | +from executorch.backends.cortex_m.passes.cortex_m_pass_manager import CortexMPassManager |
| 19 | +from torch.export import export |
| 20 | +from torchao.quantization.pt2e.quantize_pt2e import convert_pt2e, prepare_pt2e |
| 21 | +from executorch.exir import to_edge_transform_and_lower, EdgeCompileConfig |
| 22 | + |
| 23 | +quantizer = CortexMQuantizer() |
| 24 | +captured = export(model, example_inputs).module() |
| 25 | +prepared = prepare_pt2e(captured, quantizer) |
| 26 | +prepared(*example_inputs) # calibration |
| 27 | +quantized = convert_pt2e(prepared) |
| 28 | + |
| 29 | +exported = export(quantized, example_inputs) |
| 30 | +edge = to_edge_transform_and_lower( |
| 31 | + exported, |
| 32 | + compile_config=EdgeCompileConfig(_check_ir_validity=False), |
| 33 | +) |
| 34 | +edge._edge_programs["forward"] = CortexMPassManager( |
| 35 | + edge.exported_program(), CortexMPassManager.pass_list |
| 36 | +).transform() |
| 37 | +et_program = edge.to_executorch() |
| 38 | +``` |
| 39 | + |
| 40 | +In tests, `CortexMTester` wraps this pipeline: |
| 41 | + |
| 42 | +```python |
| 43 | +from executorch.backends.cortex_m.test.tester import CortexMTester |
| 44 | + |
| 45 | +tester = CortexMTester(model, example_inputs) |
| 46 | +tester.quantize().export().to_edge().run_passes().to_executorch() |
| 47 | +``` |
| 48 | + |
| 49 | +## Key Files |
| 50 | + |
| 51 | +| File | Purpose | |
| 52 | +|------|---------| |
| 53 | +| `backends/cortex_m/quantizer/quantizer.py` | `CortexMQuantizer` — quantizes model for CMSIS-NN | |
| 54 | +| `backends/cortex_m/passes/cortex_m_pass_manager.py` | `CortexMPassManager` — rewrites ATen ops → `cortex_m::` ops | |
| 55 | +| `backends/cortex_m/test/tester.py` | `CortexMTester` — test harness with `test_dialect()` and `test_implementation()` | |
| 56 | +| `backends/cortex_m/ops/operators.py` | Python op definitions and reference implementations (`cortex_m::` namespace) | |
| 57 | +| `backends/cortex_m/ops/operators.yaml` | C++ kernel registration schemas (used by build system) | |
| 58 | + |
| 59 | +C++ kernels calling CMSIS-NN APIs live under `backends/cortex_m/ops/`. |
| 60 | + |
| 61 | +## Testing |
| 62 | + |
| 63 | +**Toolchain setup (required for `test_implementation` tests):** |
| 64 | +```bash |
| 65 | +./examples/arm/setup.sh --i-agree-to-the-contained-eula |
| 66 | +source ./examples/arm/arm-scratch/setup_path.sh |
| 67 | +``` |
| 68 | + |
| 69 | +**Run all tests:** |
| 70 | +```bash |
| 71 | +source ./examples/arm/arm-scratch/setup_path.sh |
| 72 | +pytest backends/cortex_m/test/ |
| 73 | +``` |
| 74 | + |
| 75 | +`test_dialect_*` tests verify graph correctness (pure Python, no toolchain needed). |
| 76 | +`test_implementation_*` tests verify numerical accuracy on the Corstone-300 FVP (requires toolchain on PATH). |
| 77 | + |
| 78 | +**Baremetal build:** |
| 79 | +```bash |
| 80 | +backends/cortex_m/test/build_test_runner.sh |
| 81 | +``` |
| 82 | + |
| 83 | +## Adding a New Op |
| 84 | + |
| 85 | +1. Define the op schema, meta function, and reference implementation in `operators.py` |
| 86 | +2. Write the C++ kernel in `backends/cortex_m/ops/` calling CMSIS-NN APIs |
| 87 | +3. Register the `.out` kernel in `operators.yaml` |
| 88 | +4. Add a pass to rewrite the ATen op → `cortex_m::` op |
| 89 | +5. Test with `CortexMTester.test_dialect()` (graph correctness) and `test_implementation()` (numerical accuracy on FVP) |
0 commit comments