Motion Capture from Video
Video can record human and animal motion of greater variety than
can be recorded in motion capture studios - such as children playing
in the surf and cats hunting in the wild. We are exploring methods
that combine computer vision techniques with interactive
manipulation to extract arbitrary motion from single-camera video.
Our long-term goal is to develop a library of motion that
can be re-used in different environments and with different
creatures.
We suggest you use "play every frame" when running the animations.
Tracking with Active Contours
Our active contours (fauna snakes) represent image features.
They react with
image and model and are easily manipulated.
The image shows five contours and a
stick figure model tracking a boy in the surf. (Right arm
and head not tracked.) Tracking
was done on an edge-detected version (below); green
brings out contours. Tracking such a complex figure
and motion required considerable user interaction.
Click
here
for animation (1 Mg).
Automatic Model Positioning
Kinematic adjustment automatically brings the model into
alignment with contours in other frames.
(Larger View)
Active contours may
translate the body and/or rotate segments in or out of the
image plane, or about Euler axes.
Blue lines go to the anchor segment; yellow indicate the difference
between contour positions in a frame and their anchors.
Click
here for animation (1 Mg).
At left, we show model with ellipsoids for a three-dimensional
effect. The image plane is at an angle.
Click for
animation (1 Mg) .
Showhorse Trotting
The trot of an American Showhorse is extracted using 8 active contours,
6 of which act on multiple segments using inverse kinematics.
On the first frame was anchored, but the model was positioned
initially in several keyframes.
Click for animations viewed from the front
[small (0.8 Mg) ,
or large (2.5 Mg) ],
and at an angle
[small (0.8 Mg) ,
or large (2.5 Mg) ].
Stallion Galloping
The gallop of an Arabian stallion is extracted using
seven contours, all applied with inverse kinematics.
The motion is at an angle to
the image plane and non-planar.
Click for animations with anchors
[small (1.1 Mg) ,
or large (3.1 Mg) ],
and of the model only
[small (1.0 Mg) ,
or large (2.7 Mg) ].
Yearling Jumping
This yearling arab filly was captured using a combination
of automatic adjustment and user interaction.
Small (0.6 Mg). or
Large animation (1.9 Mg).
Yearling Walking
The same foal walking.
Small (0.4 Mg). or
Large animation (1.2 Mg).
Adjustment Animated
This dance motion is mostly planar and parallel to
the image plane. Tracking is relatively easy
and unambiguous, though user manipulation was still required.
Click here (1.2 Mg)
or here (3.2 Mg)
for an animation of the fauna snakes pulling
the figure into position.
Click here (1 Mg)
for an animation of the ellipsoidal figure.
