Google Play icon

Fused Reality: Making the Imagined Seem Real

Share
Posted September 29, 2015

fr_scott_howe

Dave Fedors wears the Fused Reality helmet while flying the Gippsland GA-8 Airvan. NTPS instructor Bryan Olson, in the left seat, served as safety pilot. Credits: NASA Photo / Carla Thomas

Dave Fedors wears the Fused Reality helmet while flying the Gippsland GA-8 Airvan. NTPS instructor Bryan Olson, in the left seat, served as safety pilot.
Credits: NASA Photo / Carla Thomas

There are two ways for pilots to gain proficiency in an airplane and evaluate its handling qualities. The first is to climb into the cockpit and takeoff; the second is to practice in a ground-based simulator. Each method has advantages and limitations. Now, it is possible to combine the best of both through an exciting new technology known as Fused Reality that is being tested by researchers from NASA’s Armstrong Flight Research Center at Edwards, California, and the National Test Pilot School (NTPS) in nearby Mojave.

The term Fused Reality was coined by Ed Bachelder, the system’s inventor and technical director at Systems Technology, Inc. (STI), Hawthorne, California, when the company began development in 2003 under a Phase I Small Business Innovation Research (SBIR) program for Naval Air Systems Command and the US Army. The patented technology combines real world video with interactive computer generated environments to create a highly immersive training experience for practicing complex tasks such as landing, flying in formation with other aircraft, and aerial refueling. While flying, the pilot wears a special helmet with an optical system that combines the real out-the-window view from a camera with computer-generated graphics of an airfield or another aircraft.

 In 2012, using SBIR funds, researchers from NASA and STI conducted three evaluation flights on Calspan Corporation’s highly modified Learjet in-flight simulator at Mojave. Armstrong research pilots Troy Asher and Jim Less felt that although the Fused Reality system showed great promise for flight-test and training, there were elements that needed improvement. Additional efforts, paid for through Armstrong’s Center Innovation Fund, subsequently allowed researchers to correct minor problems and add enhancements. In the next phase of flight-testing, beginning in September 2014, the team successfully demonstrated an improved Fused Reality system on board a Gippsland GA-8 Airvan research aircraft owned by the NTPS. A series of nine flights culminated in January 2015 with four flown by Armstrong research pilots Tim Williams, Dave Fedors, Scott Howe, and Wayne Ringelberg. Each pilot performed a series of tasks generated by the Fused Reality system and subjectively rated the airplane’s handling qualities. These tasks included landing on a simulated runway at altitude, formation flight, and aerial refueling drogue tracking.

Fused Reality displays a virtual environment beyond the edge of the real control panel. For aerial refueling practice the pilot attempts to connect a virtual receiver probe into a drogue receptacle extended from a computer-generated tanker. Credits: NASA Photo

In 2012, using SBIR funds, researchers from NASA and STI conducted three evaluation flights on Calspan Corporation’s highly modified Learjet in-flight simulator at Mojave. Armstrong research pilots Troy Asher and Jim Less felt that although the Fused Reality system showed great promise for flight-test and training, there were elements that needed improvement. Additional efforts, paid for through Armstrong’s Center Innovation Fund, subsequently allowed researchers to correct minor problems and add enhancements.

In the next phase of flight-testing, beginning in September 2014, the team successfully demonstrated an improved Fused Reality system on board a Gippsland GA-8 Airvan research aircraft owned by the NTPS. A series of nine flights culminated in January 2015 with four flown by Armstrong research pilots Tim Williams, Dave Fedors, Scott Howe, and Wayne Ringelberg. Each pilot performed a series of tasks generated by the Fused Reality system and subjectively rated the airplane’s handling qualities. These tasks included landing on a simulated runway at altitude, formation flight, and aerial refueling drogue tracking.

Although many ground-based flight simulators include full-motion capabilities to reproduce what it feels like to be in a real aircraft, they lack some of the cues that come with that experience. Fused Reality provides a more realistic experience because the aircraft is real and only some external elements are virtual. Scott Howe was quite impressed with his experience flying formation with a simulated KC-135.

In stenciling mode, a virtual tanker has been cut out and displayed over a real scene of the outside world. Credits: NASA Photo

In stenciling mode, a virtual tanker has been cut out and displayed over a real scene of the outside world.
Credits: NASA Photo

“I’m seeing the real world through my camera, so I’m seeing mountains and clouds and the aircraft control panel, but I’m flying formation with a virtual tanker,” he said. “I was just trying to keep station with that tanker and practice aerial refueling with the [Fused Reality] system.”

Significantly, the evaluation pilots noted that this system did not interfere in any way with obtaining actual handling qualities of the aircraft.

“I think what you gain here,” said Howe, “is the benefit of taking the simulator into the air, where you are exposed to the actual flying environment, but with the ability to superimpose a realistic simulation on top of it.”

For test pilots, Fused Reality can be used to develop handling qualities evaluation tasks for rating various aircraft configurations, advanced flight control law algorithms, pilot displays and aircraft modifications. The system can also be used to train test pilots how to do these evaluations.

The next phase of testing will be integrated into the curriculum of the Air Force Test Pilot School’s Test Management Program in March 2016. Student test pilots will design and execute a flight test program using the school’s C-12 (Beechcraft King Air), and compare ratings of conventional handling qualities tasks with results acquired using the Fused Reality system.

cott Howe performs a task with the Fused Reality system, which displays virtual objects such as runways or other aircraft over what is really there. Credits: NASA Photo / Lori Losey

cott Howe performs a task with the Fused Reality system, which displays virtual objects such as runways or other aircraft over what is really there.
Credits: NASA Photo / Lori Losey

“Fused Reality allows all pilots to learn how to fly difficult and dangerous tasks such as aerial refueling, aircraft carrier landing, formation flight and aerial firefighting, which are usually taught in a ground based flight simulator, by putting the simulator in flight in the actual aircraft,” said Armstrong project manager Bruce Cogan. “Virtual images of runways, aircraft carriers, and tanker aircraft are presented to the pilot in a helmet mounted display that reacts with the actual dynamics of the aircraft being flown.”

Cogan also noted that the Navy is investigating the use of Fused Reality for aircraft carrier landing training, and that NASA is looking into potential use of the system to enhance astronaut training.

Source: NASA

Featured news from related categories:

Technology Org App
Google Play icon
85,440 science & technology articles

Most Popular Articles

  1. New treatment may reverse celiac disease (October 22, 2019)
  2. "Helical Engine" Proposed by NASA Engineer could Reach 99% the Speed of Light. But could it, really? (October 17, 2019)
  3. New Class of Painkillers Offers all the Benefits of Opioids, Minus the Side Effects and Addictiveness (October 16, 2019)
  4. The World's Energy Storage Powerhouse (November 1, 2019)
  5. Plastic waste may be headed for the microwave (October 18, 2019)

Follow us

Facebook   Twitter   Pinterest   Tumblr   RSS   Newsletter via Email