Detecting Gestures from the Motion of the Body in Relation to Itself

Measuring the motion of a skeleton's joints in relation to each other generates signals that might be recognized as gestures.

Depth cameras, such as the Intel RealSense D415, can be configured to generate real-time skeleton data. In the video below a number of triangles are defined from the skeleton joints generated by a Microsoft Kinect.

The relative change of the shape and the orientation of a triangle can be approximated by keeping track of the Euler line, a line drawn between the circumcenter and centroid of the triangle.

In the video below, the Euler lines of the skeleton joint triangles are tracked as the measured body moves.

Calculating the relative orientation change of the cross product of Euler line pairs might be a good signal for gesture detection. An Savistsky-Golay filter can help smooth data captured by the depth camera.

The video below shows the curves generated by tracking relative motion:

Pattern detection algorithms should be able to differentiate between the measurement of motion into specific actions that could be defined by a person for themselves rather than by a designer for everyone.