Natural Navigation User Interface (NUI) 3D Demo

Using LEAP Motion for a natural user interfaces navigation through a human heart. A Master of Science in Biomedical Communications MRP Project by Brendan Polley.

Text by Brendan Polley:
“Demo of my Master of Science in Biomedical Communications MRP using the Leap Motion Controller

The development of natural user interfaces (NUIs) represents an exciting new frontier for interacting with complex three-dimensional (3D) virtual environments. Touch-free NUIs use highly sensitive cameras to track body movements in 3D space, replacing the need for keystrokes and mouse-clicks with intuitive gestures. For biocommunicators, NUIs offer a solution to unique design challenges associated with developing 3D visualizations of anatomy.

In any 3D virtual environment, selecting objects with a mouse becomes difficult as the 3D scene becomes densely populated and structures are occluded. In industries such as game development, designers can work around this limitation by creating open environments with large amounts of space between objects. Medical illustrators are not afforded the same luxury since models of anatomy represent environments in which there is no true empty space. Organs, vessels, and nerves always sit flush with adjacent structures. Consequently, accessing occluded objects requires a user to either distort his or her point of view, or remove obstructions entirely. Although individual structures can be observed, important information about the spatial relationships between different anatomic structures is lost.

For students in undergraduate anatomy courses, understanding spatial relationships is an essential learning goal. Traditionally, these relationships have been taught using dissections, however, rising costs and decreased availability of specimens have led many educational institutions to transition into using interactive, virtual anatomic models. Furthermore, even when dissections are available, spatial information of deep structures, such as the vessels of the circulatory system, are often impossible to preserve.

The goal of my project is to develop a NUI-based educational tool to help undergraduate anatomy students understand the spatial relationships of the circulatory system. By integrating a gesture-based NUI with accurate 3D anatomic models, students will have the freedom to explore individual structures while maintaining the fidelity of spatial information.”