A Mixed Reality Simulation Tool for In-Flight Evaluations (STI Paper Series)

For more than a decade, the promise of augmented, virtual, and mixed reality technologies has been to provide the means to enhance training, while significantly reducing costs. While incremental improvements have been made, it is the recent advances in high capability hardware such as stereo cameras, head-mounted displays, and motion tracking that bridge the technology gap encountered in previous approaches. Furthermore, these technologies are now available commercial-off-the-shelf (COTS) at much lower cost than a decade ago. Using this new low-cos hardware, Systems Technology, Inc. (STI) has developed Fused Reality® Flight, a blended-reality solution for in-flight pilot evaluations and training based on STI’s patented Fused Reality® technology. Building upon successful in-flight Fused Reality® demonstrations conducted for NASA, the current in-flight system applies new low-cost COTS hardware to create an in-flight experience that allows for virtual objects including other aircraft to be placed out-of-the-cockpit-window as part of combined virtual and real-world video scene viewed through a head-mounted display.