Reduce memory access and bounds checks in CPU Col2im

[Stern (Hosi121)]

Description

Reduce memory access and bounds checks in the FP32 CPU NCHW Col2im function used by ConvTranspose.

• For a 2x2 kernel with stride 2, zero padding, dilation 1, and even output dimensions, write adjacent output values together. Each output has one source value. This removes the separate zero fill and output read. Keep the addition to positive zero to preserve signed zero.
• For padding or dilation, calculate the valid source columns before the row loop. Remove the bounds check from the inner loop. This also permits SIMD for stride 1. Keep the accumulation order for each output.

The change keeps the existing GEMM, thread settings, model, and API. It adds no allocation. The full GEMM result buffer remains allocated.

Motivation and Context

The PP-OCRv6 tiny detector uses two 2x2, stride-2 ConvTranspose nodes. Their weights have shapes [16,16,2,2] and [16,1,2,2]. A runtime profile confirms that both nodes use the CPU implementation.

The table shows time reductions against main at 09dfa6ad06ed8072b2fbe57687d4f71c7914b025. Each value is the median of five comparisons with new process pairs. The operator rows include GEMM, Col2im, bias, and session overhead. The model rows include the full detector session.

Case	1 thread	4 threads
ConvTranspose: 16 input/output channels, 184x184 input, kernel 2, stride 2	38.7%	34.1%
ConvTranspose: 16 input/output channels, 64x64 input, kernel 3, stride 1, padding 1	36.9%	47.0%
ConvTranspose: 16 input/output channels, 64x64 input, kernel 2, stride 1, dilation 2	35.3%	34.2%
PP-OCRv6 tiny detector, 736x736	6.1%	7.2%
PP-OCRv6 tiny detector, 1024x1024	3.8%	7.0%

Measured on an Intel Core Ultra 7 255H under Linux x86-64/WSL2. Both source builds use GCC 13.3, Release -O3, and CPU EP. They do not use -march=native or fast math. The C++ runner times Session::Run. Both processes have the same CPU affinity, full graph optimization, and thread settings. Worker spinning is disabled. Timed blocks alternate A/B and B/A after warmup.

Four-thread times vary. For the unchanged-library A/A control on OCR 736, the median apparent changes were 2.1% with one thread and 1.0% with four threads. The one-cell operator case takes about 1 microsecond; the full sweep measured an increase below 0.1 microsecond. The reports include all 34 sweep configurations, all repeated measurements, other shapes, and the MobileNetV2 control. ARM and GPU performance were not measured.

Build steps, pinned inputs, all times, and raw reports.

Validation:

• Release CPU build and lint passed.
• 10 math tests and 81 ConvTranspose/Col2Im tests passed. Three tests for unavailable CUDA or DirectML configurations were skipped.
• Added tests cover 22 layouts, empty valid-column ranges, odd output dimensions, guards, special float values, groups, bias, and batches.
• All 552 full output comparisons against the changed library were bitwise equal.
• An ASan/UBSan check of the extracted function bodies passed 1,198 conditions and compared 4,066,230 values. This check uses the exact Col2im bodies with helper replacements for small inputs. It is not a full-runtime sanitizer build.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Copilot review overview

🟢 Approval recommended

The localized optimization preserves accumulation semantics and has comprehensive regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Optimizes FP32 CPU NCHW Col2im while preserving numerical behavior and memory safety.

Changes:

• Adds a specialized 2×2, stride-2 fast path.
• Hoists fallback bounds calculations outside the inner loop.
• Adds layout, special-value, grouping, bias, and batching tests.

File	Description
onnxruntime/​core/​util/​math_cpu.cc	Implements the optimized Col2im paths.
onnxruntime/​test/​framework/​math_test.cc	Adds correctness and guard-buffer tests.
onnxruntime/​test/​providers/​cpu/​nn/​conv_transpose_op_test.cc	Tests integration with grouped, batched ConvTranspose.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Stern (Hosi121)]

@microsoft-github-policy-service agree