Varjo XR-3

Modes:

• Sand Table
• OTW/NVG SIM TOD Switching
• OTW
• NVG SIM
• OTW/NVG SIM TOD Switching + Video blender PIP MWIR

Add-ons

• +symbology
• +ATBOSS

Hardware variants

• On a MR workstation
• On a primary Nucleus cluster standalone
• On a secondary Nucleus cluster standalone
• On Nucleus cluster with two headsets (1 primary, 1 secondary) sharing a reference puck

Masking Modes:

• COCKPIT_VIRTUAL
• COCKPIT_REAL
• COCKPIT_TRI_STATE
• COCKPIT_ALPHA_CHANNEL

Overview

Varjo XR-3 headsets are supported in two hardware configurations, either the Mixed Reality Workstation or with a Nucleus. Both of these platforms are designed to work with the XR-3 headset and SteamVR Base Station 2.0. This platform meets the hardware requirements for the XR-3 presented under Varjo's hardware requirements.

A Mixed Reality Workstation has two graphics cards, an A4500 for the Frontend and a A6000 for the XR-3 headset.

When using the XR-3 with a Nucleus, an A6000 is added to the Nucleus configuration and the XR-3 headset is run from that card.

• Note, the Mixed Reality Workstation can support XR-1 headsets and SteamVR Base Station 1.0. However, these products are no longer sold. Also the XR-1 requires a Thunderbolt 3.0 port which this platform does not have.
• Early in development we sold 2x 8.5 desksides with graphics upgraded to a 3090 to CH-47. This customer has both XR-1, XR-3 and JVC headsets. The Dell 7920 deskside doe not have Thunderbolt 3.0 ports (required for XR-1) so the customer was sold an adaptor card. This is not an official platform but something we need to keep in mind.

Its worth calling out a few hard requirements that cannot be ignored:

• Windows 10 OS Version 1809 or higher
• A CPU that support AVX2 operations
• NVIDIA GPU Driver - Varjo typically suggests the latest driver
• For optimal performance, a Quadro A6000 is recommended.

Tracking solutions supported

Varjo supports SteamVR Base Station 1.0 and 2.0, inside-out tracking, and third-party tracking.

• Since 1.0 base stations are not readily available we recommend 2.0 Base stations.
  • The number of base stations needed is highly dependent on the environment. 2 Base station are required and 4 base stations are recommended.
  • SteamVR Base Station 2.0 can be purchased from many vendors, for example New Egg.
• INSIDE-OUT TRACKING (BETA) is not supported.
• We are currently discussing r&d for supporting a 3rd-party tracking solution (https://ar-tracking.com/en/product-program/targets).
• Vive Tracker - We use the vive tracker as an anchor point for the world, which allows you to skip configuration. These can be purchased from many vendors, for example https://www.vive.com/us/accessory/vive-tracker/.

Controllers support

Controllers are not currently a requirement and are not supported.

Varjo/Steam Accounts

The customer is responsible for making an account with Varjo and procuring a Varjo subscription. This can be handled under Setting up Varjo subscriptions.

The customer is responsible for making an account with Steam. This can be handled here https://store.steampowered.com/join.

Configuration

Peripheral Setup Overview

xxx

Room Preparation for Mixed Reality

Better to have trackers from the top down. We use tri-pods.

Software Setup Overview

As outlined on Varjo's Software setup page, the following software must be installed and configured.

• Varjo Base
• Steam VR (i.e. Steam App + Steam account + SteamVR, online or offline version)
  
• Nova Configuration

Varjo Base.exe (version 3.6.1 or later)

The first step is to download and install the base Varjo Base software, provided here. The minimum software version required is Varjo Base 3.6.1 which added support for the required Vive Tracker.

The current validated version is 3.9.0.10.

Once the software is installed you must configure some Varjo Base settings:

Headset Tab

• Simple Rendering ON
• Hand Tracking OFF
• Resolution quality is High by default and it should remain that way. This is the setting we validate performance to.

Presentation Tab

• Optimize Performance ON

Trackers need different IDS

The remainder of the setup steps can be also found on the Varjo website.

Steam VR

offline vs. online

steam account

Tracking

Steam VR

Tracking is setup via steamVR.

XXX fill me in

NOVA Configuration

The HMD_OTW and HMD_NVG channels, defined in the default cluster config, should work for all of the above. When running a headset at 90hz and a Frontend channel at 60 or 120Hz you need to use the channel FRONTEND_ASYNC instead of the FRONTEND channel. These channel definitions can be found in:

 data/versions/baseline/cluster_template.cfg
 data/versions/baseline/clusterOnDeskside_template.cfg

Channels and Cluster Config

MR Workstation

OTW

##########################################
#Weapon 12 - OTW HMD
##########################################
Weapon12_Name                               "OTW HMD"
Weapon12_SensorIdList                       -1

INCLUDE versions/n8.0/renderNVGSimHmdVarjoNative.cfg
INCLUDE versions/n8.0/nucleus.cfg

INCLUDE versions/n8.0/renderNVGSimHmdVarjoNative.cfg
INCLUDE versions/n8.0/nucleus.cfg

INCLUDE versions/n8.0/systemFrontend.cfg
INCLUDE versions/n8.0/nucleus.cfg
INCLUDE versions/n8.0/systemFrontend90HzHmd.cfg

INCLUDE versions/n8.0/systemPager.cfg
INCLUDE versions/n8.0/nucleus.cfg
INCLUDE versions/n8.0/systemPager90HzHmd.cfg 

INCLUDE versions/n8.0/systemIsect.cfg
INCLUDE versions/n8.0/nucleus.cfg
INCLUDE versions/n8.0/systemIsect90HzHmd.cfg

Tracker

Fill me in

Cockpit Model and Mask

A cockpit model for an ownship is required as well as a mask, which allows masked areas of the cockpit to show the camera's view.

To configure the cockpit mask for an ownship add this token to the ownship's model.fx file.

STRING XR_COCKPIT_MASK path

Chroma Key

Chroma Key is used when two pilots are both using XR-3 headsets. The pilots can sit against greenscreens and Chroma Key allows us to composite their image coming from the color cameras with the cockpit mask and the simulator image.

To tune the Chroma Key values you must override the Nova Chroma Key control.

WITH DB/DB_SYSTEM/DB_SYS_DATA/HMD/VARJO_NATIVE/VarjoHmd.fx
BOOL    ENABLE_NOVA_CHROMA_KEY_CONTROL      TRUE

The Chroma Key values come from DB_SYSTEM/DB_SYS_DATA/CHROMA_KEY/ChromaKeyHSV.fx

Use the .fx Editor (Ctrl+Shift+J) to edit these values on-the-fly to configure for your green/blue screen

Fx Tuning

All Fx Parameters, description, range, default values.

Limit to useful FX tunings

Can we save load to the FX editor

Add page for FX editor usage. - link the general guide in this section.

Channel Type Considerations

Are there considerations for sensors, serialMV, etc.

Troubleshooting