A new design tool interprets hand gestures, enabling designers and artists to create and modify three-dimensional shapes using only their hands as a “natural user interface” instead of keyboard and mouse.
The tool, called Shape-It-Up, uses specialized computer algorithms and a depth-sensing camera to observe and interpret hand movements and gestures. The user creates shapes in a computer by interacting with a virtual workspace as the shape is displayed on a large-screen monitor.
“You create and modify shapes using hand gestures alone, no mouse or keyboard,” said Karthik Ramani, Purdue University’s Donald W. Feddersen Professor of Mechanical Engineering. “By bringing hands into the virtual space with a single depth camera we are able to manipulate the 3D artifacts as if they actually exist.”
Researchers call the underlying technique shape–gesture–context interplay. The tool could have applications in areas including games, architecture, art and engineering design, and also serves the emerging “creative maker” community, he said. The team will demonstrate the technology at the Maker Faire on Saturday and Sunday (May 18 and 19) in the San Mateo (Calif.) County Event Center.
“Our goal is to make the designer an integral part of the shape-modeling process during early design, which isn’t possible using current CAD tools,” Ramani said. “The conventional tools have non-intuitive and cognitively onerous processes requiring extensive training. We conclusively demonstrate the modeling of a wide variety of asymmetric 3D shapes within a few seconds. One can bend and deform them in various ways to explore new shapes by natural interactions. The effect is immediate.”
The creations can then be produced using a 3D printer.
Research findings appeared in the February issue of Computer-Aided Design magazine. The paper was co-authored by Ramani, graduate students Vinayak and Sundar Murugappan and postdoctoral researcher HaiRong Liu. The paper is available at https://engineering.purdue.edu/cdesign/wp/?p=1571
The research, funded by the National Science Foundation (NSF), addresses the limitations of conventional computer-aided design tools needed to create geometric shapes. Work to develop a model for transforming the research into market innovations was funded by the NSF’s Innovation Corps, or I-Corps, program and recently by NSF’s Accelerating Innovation Research (AIR).
The system harnesses the natural user interface of hand gestures to create and modify shapes.
“We are going from Windows icons, menus and pointers – or WIMPs – to a post-WIMP, natural user interface, or NUI,” Ramani said.
The tool is an advance over a previous version that was limited to creating “rotationally symmetric” objects, or those having the same measurements on all sides.
“This is important because many of the things designers need to create are not symmetrical,” Ramani said.
It uses the Microsoft Kinect camera, which senses three-dimensional space. The camera is found in consumer electronics games and can track a person’s body without using handheld electronics.
Researchers created advanced algorithms that recognize the hand gesture, understand that the hand is interacting with the shape and then modify the shape in response to the hand interaction.
See how the system works here.
The Purdue C Design Lab in the School of Mechanical Engineering is collaborating with a startup company, ZeroUI.
“ZeroUI and Purdue are pioneering a whole new co-innovation model for university-industry collaboration where we are applying Steve Blank’s and the NSF I-Corps customer-development process to academic research to ask the right questions and solve the right problems and helping to create high market impact,” said Raja Jasti, ZeroUI’s co-founder and CEO. “This technology is amazingly versatile with applications ranging from art, design and gaming to education.”
Blank worked with the NSF to develop the coursework for I-Corps, and in partnership with Stanford University, where Blank serves as a consulting associate professor, launched the first I-Corps course in 2011.
Filed Under: Rapid prototyping