joaopn/roberta-large-openai-detector-onnx-fp16

RoBERTa Large OpenAI Detector

This model is a FP16 optimized version of openai-community/roberta-large-openai-detector. It runs exclusively on the GPU. The speedup compared to the base ONNX and pytorch versions depends chiefly on your GPU's FP16:FP32 ratio. For more comparison benchmarks and sample code of a related model, check here: https://github.com/joaopn/gpu_benchmark_goemotions.

You will need the GPU version of the ONNX Runtime. It can be installed with

pip install optimum[onnxruntime-gpu] --extra-index-url https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/onnxruntime-cuda-12/pypi/simple/

For convenience, this benchmark repo provides an environment.yml file to create a conda env with all the requirements.