Skip to content

Fix KeyError in InsertIOQDQ pass for LLM quantization #17194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cccclai merged 7 commits into from
Feb 11, 2026
Merged

Fix KeyError in InsertIOQDQ pass for LLM quantization #17194

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5ebb788
Fix KeyError in InsertIOQDQ pass for LLM quantization
mohammed-saalim Feb 4, 2026
c932c65
Merge branch 'main' into fix-insert-io-qdq-keyerror
mohammed-saalim Feb 9, 2026
5249111
Merge branch 'main' into fix-insert-io-qdq-keyerror
mohammed-saalim Feb 10, 2026
621beaf
Merge branch 'main' into fix-insert-io-qdq-keyerror
mohammed-saalim Feb 10, 2026
4e3e05a
Merge branch 'main' into fix-insert-io-qdq-keyerror
mohammed-saalim Feb 10, 2026
f8cfad5
Merge branch 'main' into fix-insert-io-qdq-keyerror
mohammed-saalim Feb 10, 2026
b3964d7
Merge branch 'main' into fix-insert-io-qdq-keyerror
mohammed-saalim Feb 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions backends/qualcomm/_passes/insert_io_qdq.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,16 @@ class InsertIOQDQ(ExportPass):
"""

q_dq_map = {
# per tensor
# per tensor (quantize -> dequantize)
exir_ops.edge.quantized_decomposed.quantize_per_tensor.default: exir_ops.edge.quantized_decomposed.dequantize_per_tensor.tensor,
exir_ops.edge.quantized_decomposed.quantize_per_tensor.tensor: exir_ops.edge.quantized_decomposed.dequantize_per_tensor.tensor,
# per channel
# per tensor (dequantize -> dequantize, for nodes with dequantize encoding)
exir_ops.edge.quantized_decomposed.dequantize_per_tensor.default: exir_ops.edge.quantized_decomposed.dequantize_per_tensor.default,
exir_ops.edge.quantized_decomposed.dequantize_per_tensor.tensor: exir_ops.edge.quantized_decomposed.dequantize_per_tensor.tensor,
# per channel (quantize -> dequantize)
exir_ops.edge.quantized_decomposed.quantize_per_channel.default: exir_ops.edge.quantized_decomposed.dequantize_per_channel.default,
# per channel (dequantize -> dequantize, for nodes with dequantize encoding)
exir_ops.edge.quantized_decomposed.dequantize_per_channel.default: exir_ops.edge.quantized_decomposed.dequantize_per_channel.default,
Comment on lines 33 to +43
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding dequantize ops as keys in q_dq_map changes _create_node() behavior: it checks if target in self.q_dq_map to decide when to pop QCOM_QUANT_ATTRS and cast meta['val'] to the quantized dtype. After this change, inserted dequantize nodes (e.g. dequantize_per_tensor.tensor / dequantize_per_channel.default) will now satisfy that condition, causing their meta['val'] dtype to be incorrectly cast to the quantized dtype and moving QCOM_QUANT_ATTRS off the original node. The special-case should apply only to quantize ops; consider switching the check to if target in q_ops (or an explicit quantize-op set) so output dequant nodes keep float meta['val'] and don’t steal the original node’s quant metadata.

Copilot uses AI. Check for mistakes.
Comment on lines +37 to +43
Copy link

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change fixes a previously crashing edge case (nodes whose QCOM_ENCODING is already a dequantize op), but there doesn’t appear to be any unit coverage for InsertIOQDQ in backends/qualcomm/tests/. Adding a small FX graph test that sets QCOM_QUANT_ATTRS[QCOM_ENCODING] to dequantize_per_tensor.default and asserts the pass inserts the expected output dequant node (and doesn’t alter output meta['val'] dtype) would help prevent regressions.

Copilot uses AI. Check for mistakes.
}

def __init__(self, edge_program: torch.export.ExportedProgram):
Expand Down
Loading