Diffusers documentation

How to use the ONNX Runtime for inference

You are viewing v0.16.0 version. A newer version v0.31.0 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

How to use the ONNX Runtime for inference

🤗 Optimum provides a Stable Diffusion pipeline compatible with ONNX Runtime.

Installation

Install 🤗 Optimum with the following command for ONNX Runtime support:

pip install optimum["onnxruntime"]

Stable Diffusion Inference

To load an ONNX model and run inference with the ONNX Runtime, you need to replace StableDiffusionPipeline with ORTStableDiffusionPipeline. In case you want to load a PyTorch model and convert it to the ONNX format on-the-fly, you can set export=True.

from optimum.onnxruntime import ORTStableDiffusionPipeline

model_id = "runwayml/stable-diffusion-v1-5"
pipe = ORTStableDiffusionPipeline.from_pretrained(model_id, export=True)
prompt = "a photo of an astronaut riding a horse on mars"
images = pipe(prompt).images[0]
pipe.save_pretrained("./onnx-stable-diffusion-v1-5")

If you want to export the pipeline in the ONNX format offline and later use it for inference, you can use the optimum-cli export command:

optimum-cli export onnx --model runwayml/stable-diffusion-v1-5 sd_v15_onnx/

Then perform inference:

from optimum.onnxruntime import ORTStableDiffusionPipeline

model_id = "sd_v15_onnx"
pipe = ORTStableDiffusionPipeline.from_pretrained(model_id)
prompt = "a photo of an astronaut riding a horse on mars"
images = pipe(prompt).images[0]

Notice that we didn’t have to specify export=True above.

You can find more examples in optimum documentation.

Known Issues

  • Generating multiple prompts in a batch seems to take too much memory. While we look into it, you may need to iterate instead of batching.