How ONNX-Tensorrt register their Op

tags: tensorrt  onnx

Foreword

NVIDIA in open source projects,onnx-tensorrtVersion is v5.0, currently of 18.04 TX2 system Cuda10, Tensorrt5.0.26 currently only supports this version.
tensorrt we can see how the model into a sequence of ONNX model tensorrt support. Source code, we can see there are four Op: Split, FancyActivation, InstanceNormalization, ResizeNearest are later registered support, not tensorrt internal supports.
so we register their own custom Op, op may need to consult on how to achieve these registration functions.

Register your own custom Op: dcn_v2_forward

DCN V2 is deformable convolution of the latest version implements, DCN has many excellent characteristics, in particular target detection algorithm, adding DCN can effectively improve the detection rate. DCNV2 the current version of the source https://github.com/CharlesShang/DCNv2, currently has Cuda accelerate, so we DCN V2 as a custom Op registered to ONNX-Tensorrt in.

Sign up concrete steps

  1. Write source code dcn_v2_forward.hpp and dcn_v2_forward.cu, there is dcn_v2_forward of Op concrete realization, heavy onnx2trt :: Plugin, note that this is different from the tensorrt caffe registered layer or layers is tensorrt register themselves.
class dcn_v2_forwardPlugin final : public onnx2trt::Plugin 

Specific reference may be written in step Split, FancyActivation, InstanceNormalization, to achieve the ResizeNearest.
3. In builtin_plugins.cpp rearmost, as registration of the following is added as a dcn_v2_forward.

REGISTER_BUILTIN_PLUGIN("FancyActivation",       FancyActivationPlugin);
REGISTER_BUILTIN_PLUGIN("ResizeNearest",         ResizeNearestPlugin);
REGISTER_BUILTIN_PLUGIN("Split"        ,         SplitPlugin);
REGISTER_BUILTIN_PLUGIN("InstanceNormalization", InstanceNormalizationPlugin);
REGISTER_BUILTIN_PLUGIN("dcn_v2_forward",        dcn_v2_forwardPlugin);

REGISTER_BUILTIN_NVPLUGIN("Concat", ConcatPlugin);
  1. The core of the steps, builtin_op_importers.cpp used DEFINE_BUILTIN_OP_IMPORTER added using their own registration Op, only take effect here, use this procedure onnx2trt time to find our own custom Op.

Intelligent Recommendation

[Environment Construction] onnx-tensorrt

1 Introduction onnx-tensorrt is a model inference framework. 2. Installation 2.1 onnx-tensorrt 2.2 protobuf, version>=3.8.x 2.3 python installation...

TENSORRT, ONNX, ONNX2TENSORT installation

tensorrt:TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-10.2.cudnn8.1 onnx:onnx-1.8.1 onnx2tensorrt:onnx-tensorrt-release-7.2.1 Reference connection # ONNX installation # NX2tensorrt installation Succe...

ONNX format Tensorrt

The code written by ONNX TRT yourself. Record a few points here: The input of the model must be a Numpy format, so the data taken directly from DataLoader is not directly thrown into the model. When t...

Onnx-Tensorrt iPluginv2 SupportsFormat

Function explanation: ONNX-TENSORRT IPLUGINV2 SUPPORTSFORMAT Foreword It has been doing Huawei's clouds in this six years, which has stayed at 5.1.5 for Tensorrt. Just now have a chance, follow the on...

ONNX to TENSORRT model error

Report error message: Solution: This is an error here is CUDA10.2 existthisDownload patch 1 and install it...

More Recommendation

yolov5-V6 ->ONNX ->TensorRT

yolov5-V6 ->ONNX->TensorRT: ONNX final result is correct Tensorrt final result is incorrect solution Generate only the characteristic map, no subsequent DETECT () module 1.yolo.py Simplification...

ONNX to Tensorrt error solving

An error is as follows: solve: This problem occurs under Tensorrt8 CUDA10.2 Download two Patch patch CUDA Toolkit 10.2 Download | NVIDIA Developer   After download, Chmod 777 CUDA_XXXXX.RUN gives...

Darknet darknet to onnx to Tensorrt

Turn the weight of the darknet format to the onnx format, and then convert it to a Tensorrt format First prepare a darknet format weight file (.weights), configuration file (.cfg), the training set na...

Pytorch-Onnx-TensorRT

Tensorrt strictly depends on hardware conditions, so it is strongly related to the CUDA and CUDNN versions. Following are the four steps for this example application: Convert the pretrained image segm...

Ubuntu20.04 Install onnx-Tensorrt

1. Cloning project After cloning, the content in the onnx-Tensorrt/Third_Party/Onnx is empty. You need to download the onnx1.6 matching with Tensorrt7.1. Place the downloaded content, decompress it to...

Copyright  DMCA © 2018-2026 - All Rights Reserved - www.programmersought.com  User Notice

Top