Skip to content

Intelligent Video Analytics toolkit based on different inference backends.

License

Notifications You must be signed in to change notification settings

QuantumLiu/OpenIVA

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

47 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

English | 中文

OpenIVA

alt OpenIVA

OpenIVA is an end-to-end intelligent video analytics development toolkit based on different inference backends, designed to help individual users and start-ups quickly launch their own video AI services.
OpenIVA implements varied mainstream facial recognition, object detection, segmentation and landmark detection algorithms. And it provides an efficient and lightweight service deployment framework with a modular design. Users only need to replace the algorithm model used for their own tasks.

Features

  1. Common mainstream algorithms
  • Provides latest fast accurate pre-trained models for facial recognition, object detection, segmentation and landmark detection tasks
  1. Multi inference backends
  • Supports TensorlayerX/ TensorRT/ onnxruntime
  1. High performance
  • Achieves high performance on CPU/GPU/Ascend platforms, achieve inference speed above 3000it/s
  1. Asynchronous & multithreading
  • Use multithreading and queue to achieve high device utilization for inference and pre/post-processing
  1. Lightweight service
  • Use Flask for lightweight intelligent application services
  1. Modular design
  • You can quickly start your intelligent analysis service, only need to replace the AI models
  1. GUI visualization tools
  • Start analysis tasks only by clicking buttons, and show visualized results in GUI windows, suitable for multiple tasks

alt Sample Face landmark alt Sample Face recognition alt Sample YOLOX

Performance benchmark

Testing environments

  • i5-10400 6c12t
  • RTX3060
  • Ubuntu18.04
  • CUDA 11.1
  • TensorRT-7.2.3.4
  • onnxruntime with EPs:
    • CPU(Default)
    • CUDA(Manually Compiled)
    • OpenVINO(Manually Compiled)
    • TensorRT(Manually Compiled)

Performance

Facial recognition

Run
python test_landmark.py
batchsize=8, top_k=68, 67 faces in the image

  • Face detection
    Model face_detector_640_dy_sim

    onnxruntime EPs FPS faces per sec
    CPU 32 2075
    OpenVINO 81 5374
    CUDA 105 7074
    TensorRT(FP32) 124 7948
    TensorRT(FP16) 128 8527
  • Face landmark
    Model landmarks_68_pfld_dy_sim

    onnxruntime EPs faces per sec
    CPU 69
    OpenVINO 890
    CUDA 2061
    TensorRT(FP32) 2639
    TensorRT(FP16) 3131

Run
python test_face.py
batchsize=8

  • Face embedding
    Model arc_mbv2_ccrop_sim

    onnxruntime EPs faces per sec
    CPU 212
    OpenVINO 865
    CUDA 1790
    TensorRT(FP32) 2132
    TensorRT(FP16) 2812

Objects detection

Run
python test_yolo.py
batchsize=8 , 4 objects in the image

  • YOLOX objects detect
    Model yolox_s(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 9.3 37.2
    OpenVINO 13 52
    CUDA 77 307
    TensorRT(FP32) 95 380
    TensorRT(FP16) 128 512

    Model yolox_m(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 4 16
    OpenVINO 5.5 22
    CUDA 46.8 187
    TensorRT(FP32) 64 259
    TensorRT(FP16) 119 478

    Model yolox_nano(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 47 188
    OpenVINO 80 320
    CUDA 210 842
    TensorRT(FP32) 244 977
    TensorRT(FP16) 269 1079

    Model yolox_tiny(ms_coco)

    onnxruntime EPs FPS Objects per sec
    CPU 33 133
    OpenVINO 43 175
    CUDA 209 839
    TensorRT(FP32) 248 995
    TensorRT(FP16) 327 1310

Progress

  • Multi inference backends

    • onnxruntime
      • CPU
      • CUDA
      • TensorRT
      • OpenVINO
    • TensorlayerX
    • TensorRT
  • Asynchronous & multithreading

    • Data generate threads
    • AI compute threads
    • Multifunctional threads
    • Collecting threads
  • Lightweight service

    • prototype
  • GUI visualization tools

  • Common algorithms

    • Facial recognition

      • Face detection

      • Face landmark

      • Face embedding

    • Object detection

      • YOLOX
    • Semantic/Instance segmentation

    • Scene classification

      • prototype
  • Data I/O

    • Video decoding
      • OpenCV decoding
        • Local video files
        • Network stream videos
    • Data management
      • Facial identity database
      • Data serialization

About

Intelligent Video Analytics toolkit based on different inference backends.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages