Augmented Reality Controller Puts Science in Students’ Hands

STEM educators are always keen to put their subjects in front of students at an early age. One of the biggest tools in teaching children about science and engineering are visuals, where kids can see what concepts are being presented. In an example of new technologies introduced to elementary schools, a virtual reality tool puts 3D objects like the human heart, fossils or galaxies in the hands of young students.
 
The Merge Cube is a soft block of foam inscribed with raised patterns that interacts with an app and the camera of a computer, tablet or smartphone to show a three-dimensional image. While the cube does not have any haptic sensors, users do get a “feel” for the object while they turn or manipulate the six-faced block.
 
Using the example of Earth, one of the programs included in its package, Merge co-founder Andrew Trickett said, “You don’t have six independent experiences on each face [of the cube]. It is as if it is one object, like I am holding the Earth in my hand. As you rotate it smoothly, it gives the illusion that you are holding the Earth. As you push it away from you the object gets smaller. Pull it toward you and it gets bigger.”
 
The system uses computer vision to scan the raised inscriptions on the sides of the cube, which are illuminated with a special reflective paint needed to provide points of contrast. The app then determines how to use that information.

To use the system, the user holds the cube in front of the device’s camera. A 3D image of an object then appears on the screen. As the user rotates the cube, the image on the screen rotates, enabling a clear view of all sides and angles of the object.

Get Involved: Find the ASME Membership Level That Works for You
 
The cube originally was targeted for the consumer market, Trickett said. The company became aware of the potential in educational markets after receiving feedback from teachers. “It kind of built up through the grassroots level,” he said.
 
Merge had to build more technical content to offer the educational market, as well as teaching plans. “This has tremendous promise as an educational asset, but we had to have more truly educational content, particularly aligning standards for STEM for curriculum,” he added.
 
Although the cube does not include haptics, it is engineered to have the same tensile strength as the palm of a hand, Trickett said. “It feels very intuitive, the digital object feels like a natural part of your hand,” he added.

Based on Your Interests: Engineering Gift Guide
 
High-quality foam is essential for providing the “feel,” but Trickett said precise manufacturing is key to the cube’s performance.
 
“Manufacturing of the patterns, and particularly around the edges, is the tricky part,” Trickett said. “That’s where you’re rotating from pattern to pattern. At the sub millimeter level.”
 
The cube also works with headsets, sold separately. The headset, of course, provides a hands-free experience. The smartphone screen resides inside the headset, covering the user’s field of view. But the camera projects a real-world scene, and gives a more immersive experience.
 
The stamping feature allows students to move around the animated 3D object, providing the same interactions without the cube in view. It offers many content cards that read content in multiple languages, fonts, color, and sizes through Microsoft’s Immersive Reader application.

Engineering for Kids: Geeks for Kids Gives Power to Play
 
Merge EDU is a digital learning platform that engages kids in hands-on activities. It works with most iOS and Android phones or tablets, and some Chromebook devices. The apps are designed to provide new ways to teach scientific concepts using the cube. Merge apps are free, but additional features require a subscription.
 
The Cube costs $19.99. The headset costs $69.99.
 
John Kosowatz is senior editor at ASME.