Proto TAI++: Exploring Perceptually Consistent Creation of Planar Shape Assemblies Through Multimodal Tangible Interactions

Abstract

We explore tangible 3D interactions that allow for geometric and perceptual correspondence between a midair modality and the 3D elements it controls. To demonstrate our approach, we use a concrete application scenario through ProtoTAI++, a multimodal system using penbased drawing of planar shapes and their subsequent midair assembly via a handheld planar proxy. The planarity of the proxy is a key element that physically embodies virtual planar shapes during 3D manipulation. We also investigate advanced operations such as patterning and blending using asymmetric bimanual interactions that augment midair motion with multitouch gestures. We describe a threestage user evaluation with our system wherein our goal is to (a) study the effectiveness of the planar proxy as a tangible 3D modality, (b) contrast key features of our approach with a GUIbased planar shape assembly system, and (c) evaluate user experience and performance in creative tasks using ProtoTAI++.