Revision history [back]

To install TensorRT for TensorFlow on a GPU, follow the steps below:

1. Check the compatibility of the TensorRT version with the TensorFlow version and the GPU architecture.

2. Download and install the TensorRT package for your system from the NVIDIA website.

3. Install TensorFlow in your environment (e.g., virtual environment, container).

4. Install the TensorRT TensorFlow integration package using pip:

pip install tensorflow-gpu==x.x.x+nvyy.xx

where x.x.x denotes the TensorFlow version, and yy.xx denotes the TensorRT version.

1. Verify the installation by importing the TensorFlow package and checking if the TensorRT backend is available:

import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))

This should show the TensorRT backend as one of the available devices.

1. Use the TensorRT inference optimizations by converting the TensorFlow model to a TensorRT inference engine. This can be done using the TensorFlow-TensorRT conversion tools, such as the GraphSurgeon and UFF converters.

2. Run your TensorFlow code on the GPU with TensorRT speedups!