Using kinect to capture human motion for mechanism synthesis, motion generation and visualization

In this paper, we are presenting a framework for capturing human motions using Microsoft Kinect sensor for the purpose of 1) generating task positions for mechanism and robot synthesis, and 2) generation and visualization of B-spline interpolated and approximated motion from the captured task positions. The theoretical foundation of this work lies in Kinematic Mapping, Dual and Bi-quaternions, and NURBS (Non-Uniform Rational B-spline) geometry. Lately, Kinect has opened doors for creation of natural and intuitive human-machine interactive (HMI) systems in medicine, robotic manipulation, CAD, and many other fields, where visual-sensing and -capture is a central theme. Kinect has made a huge impact in physical therapy area, achieving new benchmarks in tele-rehabilitation by improving physical exercise assessment, monitoring and supervision using the skeletal data. Moreover, Kinect's depth sensing capability has helped in retrieving depth information required for robotic vision in grasping, object recognition which was previously done using computationally demanding computer vision algorithms. Kinect's point cloud data with interactive gestures has proven to be useful in various CAD software for conceptual design of shapes. Mechanism synthesis is one of the areas in Kinematics, where Kinect-provided skeletal data can be leveraged to design and develop highly customized end-user collaborated mechanism solutions. We demonstrate that using Kinect, OpenGL, and Openframeworks, we can capture discrete (or, key) rigid body displacements, continuous motions, and generate and visualize rational B-spline motions from captured key positions. Capturing only a few key positions results in significant data savings and also provides a natural way to create tasks for mechanism synthesis problems. The output is a set of dual quaternions and 4 × 4 homogeneous transforms representing a task motion, which can be used as an input for mechanism synthesis applications. The tool produced also allows users to generate trajectories of various points on a moving rigid body interactively. A Kinect-based capture of such motions can help create highly-customized assistive devices for people who suffer from a range of motion-related difficulties due to old age or disabilities.