Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real Time Applications

Abstract

Recent development of perframe motion extraction method can generate the skeleton of human motion in realtime with the help of RGBD cameras such as Kinect. This leads to an economic device to provide human motion as input for realtime applications. As generated by a singleview image plus depth information, the extracted skeleton usually has problems of unwanted vibration, bonelength variation, selfocclusion, etc. This paper presents an approach to overcome these problems by synthesizing the skeletons generated by duplex Kinects, which capture the human motion in different views. The major technical difficulty of this synthesis comes from the inconsistency of two skeletons. Our algorithm is formulated under the constrained optimization framework by using the bonelengths as hard constraints and the tradeoff between inconsistent joint positions as soft constraints. Schemes are developed to detect and reposition the problematic joints generated by perframe method from duplex Kinects. As a result, we develop an easy, cheap and fast approach that can improve the skeleton of human motion at an average speed of 5 ms per frame.