Triton Model Navigator v0.5.5

• new: Public nav.utilities module with UnpackedDataloader wrapper
• new: Added support for strict flag in Torch custom config
• new: Extended TensorRT custom config to support builder optimization level and hardware compatibility flags
• fix: Invalid optimal shape calculation for odd values in max batch size

• Version of external components used during testing:
  • PyTorch 2.1.0a0+fe05266f
  • TensorFlow 2.12.0
  • TensorRT 8.6.1
  • ONNX Runtime 1.14.1
  • Polygraphy: 0.47.1
  • GraphSurgeon: 0.3.26
  • tf2onnx v1.14.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.5.4

• new: Custom implementation for ONNX and TensorRT runners

• new: Use CUDA 12 for JAX in unit tests and functional tests

• new: Step-by-step examples

• new: Updated documentation

• new: TensorRTCUDAGraph runner introduced with support for CUDA graphs

• fix: Optimal shape not set correctly during adaptive conversion

• fix: Find max batch size command for JAX

• fix: Save stdout to logfiles in debug mode

• Version of external components used during testing:

  • PyTorch 2.1.0a0+fe05266f
  • TensorFlow 2.12.0
  • TensorRT 8.6.1
  • ONNX Runtime 1.14.1
  • Polygraphy: 0.47.1
  • GraphSurgeon: 0.3.26
  • tf2onnx v1.14.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.5.3

• fix: filter outputs using output_metadata in ONNX runners

• Version of external components used during testing:
  • PyTorch 2.0.0a0+1767026
  • TensorFlow 2.11.0
  • TensorRT 8.5.3.1
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.44.2
  • GraphSurgeon: 0.3.26
  • tf2onnx v1.14.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.5.2

• new: Added Contributor License Agreement (CLA)

• fix: Added missing --extra-index-url to installation instruction for pypi

• fix: Updated wheel readme

• fix: Do not run TorchScript export when only ONNX in target formats and ONNX extended export is disabled

• fix: Log full traceback for ModelNavigatorUserInputError

• Version of external components used during testing:

  • PyTorch 2.0.0a0+1767026
  • TensorFlow 2.11.0
  • TensorRT 8.5.3.1
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.44.2
  • GraphSurgeon: 0.3.26
  • tf2onnx v1.14.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.5.1

• fix: Using relative workspace cause error during Onnx to TensorRT conversion

• fix: Added external weight in package for ONNX format

• fix: bugfixes for functional tests

• Version of external components used during testing:

  • PyTorch 1.14.0a0+410ce96
  • TensorFlow 2.11.0
  • TensorRT 8.5.3
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.44.2
  • GraphSurgeon: 0.4.6
  • tf2onnx v1.13.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.5.0

• new: Support for PyTriton deployemnt

• new: Support for Python models with python.optimize API

• new: PyTorch 2 compile CPU and CUDA runners

• new: Collect conversion max batch size in status

• new: PyTorch runners with compile support

• change: Improved handling CUDA and CPU runners

• change: Reduced finding device max batch size time by running it once as separate pipeline

• change: Stored find max batch size result in separate filed in status

• Version of external components used during testing:

  • PyTorch 1.14.0a0+410ce96
  • TensorFlow 2.11.0
  • TensorRT 8.5.3
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.44.2
  • GraphSurgeon: 0.4.6
  • tf2onnx v1.13.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.4.4

• fix: when exporting single input model to saved model, unwrap one element list with inputs

• Version of external components used during testing:
  • PyTorch 1.14.0a0+410ce96
  • TensorFlow 2.11.0
  • TensorRT 8.5.3
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.44.2
  • GraphSurgeon: 0.4.6
  • tf2onnx v1.13.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.4.3

• fix: in Keras inference use model.predict(tensor) for single input models

• Version of external components used during testing:
  • PyTorch 1.14.0a0+410ce96
  • TensorFlow 2.11.0
  • TensorRT 8.5.3
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.44.2
  • GraphSurgeon: 0.4.6
  • tf2onnx v1.13.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.4.2

• fix: loading configuration for trt_profile from package
• fix: missing reproduction scripts and logs inside package
• fix: invalid model path in reproduction script for ONNX to TRT conversion
• fix: collecting metadata from ONNX model in main thread during ONNX to TRT conversion

• Version of external components used during testing:
  • PyTorch 1.14.0a0+410ce96
  • TensorFlow 2.11.0
  • TensorRT 8.5.3
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.44.2
  • GraphSurgeon: 0.4.6
  • tf2onnx v1.13.0
  • Other component versions depend on the used framework containers versions.
    See its support matrix
    for a detailed summary.

Triton Model Navigator v0.4.1

• fix: when specified use dynamic axes from custom OnnxConfig

• Version of external components used during testing:
  • PyTorch 1.14.0a0+410ce96
  • TensorFlow 2.11.0
  • TensorRT 8.5.2.2
  • ONNX Runtime 1.13.1
  • Polygraphy: 0.43.1
  • GraphSurgeon: 0.4.6
  • tf2onnx v1.13.0
  • Other component versions depend on the used framework and Triton Inference Server containers versions.
    See its support matrix
    for a detailed summary.