perf(trt): optimize YOLOv26 preprocessing pipeline

[HaoyangZhuo]

Summary

• add fused CUDA preprocessing for the TensorRT YOLOv26 detector
• reuse host and device buffers across inference calls
• use page-locked host memory when available
• transfer the compact raw BGR image instead of the FP32 CHW tensor
• add a reproducible benchmark with stage-level timing and correctness checks
• preserve the original CPU preprocessing path for validation and fallback

Implementation

The optimized path submits the following operations to the same CUDA stream:

1. raw BGR image H2D copy
2. fused CUDA preprocessing
3. TensorRT enqueueV3()
4. output D2H copy

The fused preprocessing kernel performs resize, letterbox padding, BGR-to-RGB conversion, normalization, and HWC-to-CHW conversion, writing directly into the TensorRT input buffer.

Three execution modes are available for benchmarking:

• Baseline: CPU OpenCV preprocessing with pageable host buffers
• PinnedCpu: CPU preprocessing with reusable page-locked buffers
• Optimized: compact raw-image transfer with fused CUDA preprocessing

The default detection path uses CUDA preprocessing for CV_8UC3 input. Other supported three-channel image depths fall back to the CPU preprocessing path.

Optimization resources also degrade safely:

• failed host registration keeps valid reusable pageable buffers
• failed pinned raw-image allocation falls back to CPU preprocessing
• timing events are created only when timing is requested
• asynchronous CUDA work is drained before exception paths return

Validation

Environment:

• NVIDIA GeForce RTX 3090
• CUDA 12.8
• TensorRT 10.9.0.34
• FP32 YOLOv26n engine
• static input shape [1, 3, 640, 640]
• output shape [1, 300, 6]

Build and run:

cmake -S . -B build/yolo26-trt \
  -DENABLE_TENSORRT=ON \
  -DCUDA_DIR=/usr/local/cuda \
  -DTensorRT_DIR=/path/to/TensorRT

cmake --build build/yolo26-trt \
  --target lite_yolov26_benchmark \
  -j 1

LD_LIBRARY_PATH="${PWD}/third_party/opencv/lib:${LD_LIBRARY_PATH:-}" \
  build/yolo26-trt/install/bin/lite_yolov26_benchmark \
  build/yolo26-export/yolo26n_fp32.engine \
  examples/lite/resources/test_lite_detection_1.jpg \
  20 200

Correctness results:

• 9 preprocessing test cases passed
• 1,228,800 tensor elements checked per case
• zero mismatched elements and zero absolute error
• 200/200 detection consistency checks passed
• all three execution modes produced the same 5 detection boxes

Performance results over 200 measured iterations:

Mode	Mean latency	Throughput
Baseline	5.023380 ms	199.069166 FPS
Pinned CPU	4.627208 ms	216.113046 FPS
Optimized CUDA	2.039579 ms	490.297184 FPS

The optimized path achieved:

• 2.462949x speedup over the baseline path
• 2.268707x speedup over the pinned CPU path

Notes

• detect_with_timing() and compare_preprocess() are intended for validation and benchmarking.
• TRTYoloV26 instances retain reusable buffers and are not safe for concurrent inference calls from multiple threads.

[wangzijian1010]

You can integrate the code from benchmark.cpp into your YOLO26 test file instead of keeping it as a separate file.