ANTz Xr

ANTz + Xreality™ (extension)

"ANTz+Xr™ pre-beta software officially released for free!"

- Saxon Digital, 11 Feb. 2021.

ANTz+Xr is ANTz plus Xreality (extension) by Saxon Digital, a derivative of ANTz that supports AR, VR and Mixed Reality. A companion app to ANTz core that provides enhanced features which are not (yet) feasible for open source inclusion, (ie. utilizes 3rd party closed-source SDKs). The pre-beta version is now publicly available (in object form) to provide access to early adopters and get developer feedback. So, if you have a SpaceMouse or zSpace, or want some ultra-low-latency video IO, then you may like to try the free ANTz-Xr app!

Where were headed next:

• ANTz + PODmind™ environments for GaiaViz™ back-end system:
  • DGX SuperPOD for Enterprise.
    • IBM Reference Architecture
  • Aurora - Exascale Science for Everyone!
    • Porting to Aurora
    • Node Level Overview
    • Cray Interconnect

ANTz+Xr Hardware Support:

• AR/VR/XR devices:

  • 3Dconnexion SpaceMouse
  • zSpace AR/VR Desktop and Laptop

• Broadcast Video IO:

  • Key Features:
    • Low-latency SDI to GPU with (genlock) Sync.
      • Supports display walls at 16K+.
    • Ultra-low-latency
      • JPEG2000 encode -> transmit -> decode.
      • SDI pass-through with 3D/CG keyer.
    • 8K/UHD2, 4K/UHD or 2K/HD with 12G/6G/3G/1.5G-SDI.
      • HFR 12-bit, 10-bit or 8-bit (4:4:4 or 4:2:2) with Deep Color and HDR.
      • See Wiki - Video Specs for details.
    • Formats: J2K, ProRes, DnX, MPEG2/4, H.264/7...
    • Standards: SMPTE, DCI (option for DCP encryption).
    • GPUDirect DMA transfer between GPU-GPU and Video IO.
    • Display Wall with multi-GPU systems and/or parrallel cluster.
      • 16K Unified Video IO (all video streams available to each node).
  • Vendor Support:
    • AJA Video Systems - Kona and Corvid
      • 2 frames of latency with Sync (ingest to playout).
        • eg. Low-latency = 33.3ms (@59.94Hz).
    • Blackmagic Design - Intensity and DeckLink
      • 2 frames of latency with Sync (ingest to playout).
        • eg. Low-latency = 33.3ms (@59.94Hz).
    • IntoPIX - JPEG2000 IP-cores for FPGA
      • Sub-frame latency from SDI Ingest -> network -> SDI Playout
        • Ultra-low-latency < 3ms (all framerates).
    • NVIDIA
      • GPUDirect + TPU + GPU + Sync**
        • Tesla and Quadro 4000+ (series) Keplar, Maxwell, Pascal and Turing (RTX).
        • Quadro Sync (series) provides GPU genlock.
      • Mellanox Ethernet and InfiniBand End-to-End HPC (40/56/100/200 Gb/s).

Our strategy is to steadily migrate Xreality methods into the open source! In general, their will be an ongoing desire and real world need to deploy closed source technologies. Xreality will continue to serve ANTz as a bridge to closed domains and focus on tech that shows potential of becoming open source compatible.

If you need Xreality customization (eg. new devices), or require access to the source, then feel free to contact the Xreality team at: https://saxondigital.com/contact.html

ANTz+Xr under consideration:

• AR/VR/XR devices:
  • Looking Glass Factory - Holographic Display
  • Microsoft Halolens II
  • Microsoft kinect 2
  • Sony Spacial Reality Display
  • Tactical Haptics - Reactive Grip
• Ultra-low-latency Video IO:
  • Matrox Imaging and Video
  • Bluefish444
  • AMD Radeon Pro + S400 Sync + DirectGMA

Our (current) migration strategy from closed to open source:

• 3Dconnexion SpaceMouse now works using our custom freeglut.dll patch with HID.
  • ANTz core can support most buttons with our direct HID implementation.
    • But not the Dual-Function key events, nor the LCD menu.
  • ANTz+Xr supports all models with all button event types and the LCD menu.
    • eg. SpaceMouse Enterprise.
• zSpace - Planning to support OpenXR, (hopefully very soon!).
• AJA Video Systems - Discussions with them suggest that maybe we can work something out.
• Blackmagic Design - SDK can be redistributed, but only in full, (BMD SDK is 427MB vs 25MB for ANTz), but perhaps we can find a workaround to the file size problem, (imagine if every library was this big, ANTz would be 4GB+.)
• IntoPIX - Supports OpenJPEG, though software is not generally fast enough for realtime JPEG200 video, (and why we use their licensed FPGA IPcore).
• NVIDIA GPUDirect - While NVIDIA GPUDirect is open source compatible, the example source code and (required) support is contingent on a 3rd party licensing, (eg. AJA SDK).

---

Khronos

The Khronos Group leads our XR charge to open source!

Khronos helps guide us to our future ecosystem of open source Xreality tools, a path that is practical today!

Historically, they have brought us excellent graphics standards (OpenGL, GLSL, GLES, COLLADA and Vulkan), in addition to OpenCL for heterogenous computing on the CPU, GPU and FPGA.

Their newer standards include OpenVX (vision processing), glTF (supercedes COLLADA), ANNARI (data viz, but not ready yet) and NNEF for neural networks could be helpful for our Neural Physics Engine (NPE).

But, first let's not forget everyone else, see our complete list of ANTz 3rd party library dependencies .

Khronos Libraries of Interest:

• OpenVX - Vision Processing
• OpenCL - Open Compute Language
  • Parallel Programming of Heterogeneous Systems.
• SPIR - Intermediate language for Vulkan, OpenGL or OpenCL.
• SYCL - Sits on top of OpenCL and other backends.
• ANARI - Analytic Rendering Interface for Data Visualization.
  • ANARI Press Release - (2020-03-03)
• NNEF - Neural Network Exchange Format
  • OpenVX Neural Network Extension
• OpenXR - UNIFYING REALITY
  • Khronos OpenXR Concept Image
  • Companies Publicly Supporting OpenXR - (eg. zSpace).

"OpenXR is a royalty-free, open standard that provides high-performance access to Augmented Reality (AR) and Virtual Reality (VR)—collectively known as XR—platforms and devices." - The Khronos Group

We support OpenXR and encourage others to follow this path! OpenXR allows us to reach an ever growing range of 'Mixed Reality' (AR/VR/+) hardware, in a cross-platform an open source friendly manner.

---

Useful References:

"Nvidia Offers A Turnkey Supercomputer -- The DGX SuperPod." Forbes, 19 Jun. 2019. Press release.

Aurora Programming Models:

---

OpenXR API Concept: