Skip to content

[ET-VK][qconv] Add flexible layout impl for im2col#17249

Merged
meta-codesync[bot] merged 2 commits intogh/SS-JIA/411/basefrom
gh/SS-JIA/411/head
Feb 5, 2026
Merged

[ET-VK][qconv] Add flexible layout impl for im2col#17249
meta-codesync[bot] merged 2 commits intogh/SS-JIA/411/basefrom
gh/SS-JIA/411/head

Conversation

@SS-JIA
Copy link
Contributor

@SS-JIA SS-JIA commented Feb 5, 2026

Stack from ghstack (oldest at bottom):

This implements an im2col-based approach for quantized conv2d, which
transforms convolution into matrix multiplication. The im2col
transformation extracts sliding windows from the input tensor and
reshapes them into a 2D matrix, enabling reuse of the optimized
pointwise convolution shader for the compute-intensive portion.

Two im2col shaders are added:

  • q8ta_im2col.glsl: Generic shader with layout-agnostic input access
    via BufferMetadata and specialization constants
  • q8ta_im2col_4w4c.glsl: Optimized shader for 4W4C input layout that
    exploits the alignment between consecutive width positions and packed
    channel values

The im2col output is always stored in 4W4C layout to match the expected
input format of the pointwise convolution shader. The operator is
registered as etvk.q8ta_conv2d_im2col.default and currently supports
non-grouped convolutions where input channels is a multiple of 4.

Authored with assistance from Claude.

Differential Revision: D92407723

This implements an im2col-based approach for quantized conv2d, which
transforms convolution into matrix multiplication. The im2col
transformation extracts sliding windows from the input tensor and
reshapes them into a 2D matrix, enabling reuse of the optimized
pointwise convolution shader for the compute-intensive portion.

Two im2col shaders are added:
- `q8ta_im2col.glsl`: Generic shader with layout-agnostic input access
  via BufferMetadata and specialization constants
- `q8ta_im2col_4w4c.glsl`: Optimized shader for 4W4C input layout that
  exploits the alignment between consecutive width positions and packed
  channel values

The im2col output is always stored in 4W4C layout to match the expected
input format of the pointwise convolution shader. The operator is
registered as `etvk.q8ta_conv2d_im2col.default` and currently supports
non-grouped convolutions where input channels is a multiple of 4.

Authored with assistance from Claude.

Differential Revision: [D92407723](https://our.internmc.facebook.com/intern/diff/D92407723/)

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Feb 5, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/17249

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Pending, 3 Unrelated Failures

As of commit 6fa8b78 with merge base 1cffd23 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 5, 2026
SS-JIA pushed a commit that referenced this pull request Feb 5, 2026
This implements an im2col-based approach for quantized conv2d, which
transforms convolution into matrix multiplication. The im2col
transformation extracts sliding windows from the input tensor and
reshapes them into a 2D matrix, enabling reuse of the optimized
pointwise convolution shader for the compute-intensive portion.

Two im2col shaders are added:
- `q8ta_im2col.glsl`: Generic shader with layout-agnostic input access
  via BufferMetadata and specialization constants
- `q8ta_im2col_4w4c.glsl`: Optimized shader for 4W4C input layout that
  exploits the alignment between consecutive width positions and packed
  channel values

The im2col output is always stored in 4W4C layout to match the expected
input format of the pointwise convolution shader. The operator is
registered as `etvk.q8ta_conv2d_im2col.default` and currently supports
non-grouped convolutions where input channels is a multiple of 4.

Authored with assistance from Claude.

Differential Revision: [D92407723](https://our.internmc.facebook.com/intern/diff/D92407723/)

ghstack-source-id: 338601821
Pull Request resolved: #17249
@github-actions
Copy link

github-actions bot commented Feb 5, 2026

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

This implements an im2col-based approach for quantized conv2d, which
transforms convolution into matrix multiplication. The im2col
transformation extracts sliding windows from the input tensor and
reshapes them into a 2D matrix, enabling reuse of the optimized
pointwise convolution shader for the compute-intensive portion.

Two im2col shaders are added:
- `q8ta_im2col.glsl`: Generic shader with layout-agnostic input access
  via BufferMetadata and specialization constants
- `q8ta_im2col_4w4c.glsl`: Optimized shader for 4W4C input layout that
  exploits the alignment between consecutive width positions and packed
  channel values

The im2col output is always stored in 4W4C layout to match the expected
input format of the pointwise convolution shader. The operator is
registered as `etvk.q8ta_conv2d_im2col.default` and currently supports
non-grouped convolutions where input channels is a multiple of 4.

Authored with assistance from Claude.

Differential Revision: [D92407723](https://our.internmc.facebook.com/intern/diff/D92407723/)

[ghstack-poisoned]
SS-JIA pushed a commit that referenced this pull request Feb 5, 2026
Pull Request resolved: #17249

This implements an im2col-based approach for quantized conv2d, which
transforms convolution into matrix multiplication. The im2col
transformation extracts sliding windows from the input tensor and
reshapes them into a 2D matrix, enabling reuse of the optimized
pointwise convolution shader for the compute-intensive portion.

Two im2col shaders are added:
- `q8ta_im2col.glsl`: Generic shader with layout-agnostic input access
  via BufferMetadata and specialization constants
- `q8ta_im2col_4w4c.glsl`: Optimized shader for 4W4C input layout that
  exploits the alignment between consecutive width positions and packed
  channel values

The im2col output is always stored in 4W4C layout to match the expected
input format of the pointwise convolution shader. The operator is
registered as `etvk.q8ta_conv2d_im2col.default` and currently supports
non-grouped convolutions where input channels is a multiple of 4.

Authored with assistance from Claude.
ghstack-source-id: 338638552
@exported-using-ghexport

Differential Revision: [D92407723](https://our.internmc.facebook.com/intern/diff/D92407723/)
@meta-codesync meta-codesync bot merged commit 752fdb3 into gh/SS-JIA/411/base Feb 5, 2026
176 of 184 checks passed
@meta-codesync meta-codesync bot deleted the gh/SS-JIA/411/head branch February 5, 2026 23:29
SS-JIA pushed a commit that referenced this pull request Feb 6, 2026
Pull Request resolved: #17249

This implements an im2col-based approach for quantized conv2d, which
transforms convolution into matrix multiplication. The im2col
transformation extracts sliding windows from the input tensor and
reshapes them into a 2D matrix, enabling reuse of the optimized
pointwise convolution shader for the compute-intensive portion.

Two im2col shaders are added:
- `q8ta_im2col.glsl`: Generic shader with layout-agnostic input access
  via BufferMetadata and specialization constants
- `q8ta_im2col_4w4c.glsl`: Optimized shader for 4W4C input layout that
  exploits the alignment between consecutive width positions and packed
  channel values

The im2col output is always stored in 4W4C layout to match the expected
input format of the pointwise convolution shader. The operator is
registered as `etvk.q8ta_conv2d_im2col.default` and currently supports
non-grouped convolutions where input channels is a multiple of 4.

Authored with assistance from Claude.
ghstack-source-id: 338638552
@exported-using-ghexport

Differential Revision: [D92407723](https://our.internmc.facebook.com/intern/diff/D92407723/)
SS-JIA pushed a commit that referenced this pull request Feb 6, 2026
Pull Request resolved: #17249

This implements an im2col-based approach for quantized conv2d, which
transforms convolution into matrix multiplication. The im2col
transformation extracts sliding windows from the input tensor and
reshapes them into a 2D matrix, enabling reuse of the optimized
pointwise convolution shader for the compute-intensive portion.

Two im2col shaders are added:
- `q8ta_im2col.glsl`: Generic shader with layout-agnostic input access
  via BufferMetadata and specialization constants
- `q8ta_im2col_4w4c.glsl`: Optimized shader for 4W4C input layout that
  exploits the alignment between consecutive width positions and packed
  channel values

The im2col output is always stored in 4W4C layout to match the expected
input format of the pointwise convolution shader. The operator is
registered as `etvk.q8ta_conv2d_im2col.default` and currently supports
non-grouped convolutions where input channels is a multiple of 4.

Authored with assistance from Claude.
ghstack-source-id: 338638552
@exported-using-ghexport

Differential Revision: [D92407723](https://our.internmc.facebook.com/intern/diff/D92407723/)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants