Motion Caption Laboratory
If you have ever seen a making-of-video of a modern Hollywood movie, you may have come across some elements of our new motion capture laboratory. This particular technology is put to use when characters within a movie must be animated on a computer, something which is the case in most modern films in the fantasy and science fiction genres.
Small silver spheres, called “markers“, are probably the most obvious element of motion capture technology. These markers are placed upon the movable body parts of the protagonist (the participant), particularly on their joints. Either the participant wears a special suit to which the markers adhere, or the markers are glued to the body with double-sided tape. There are 12 cameras on the ceiling of the laboratory; each illuminates an area of about five square meters with infra-red light. The reflecting surface of the markers makes them particularly visible to the cameras. If a marker is captured by two or more of the cameras, the system can identify the exact location of the marker. All information captured by the system which lies below a certain threshold is discarded, meaning that all that remains is a cloud of white points on a black background. The combination of recordings of several cameras along with information regarding camera geometrics and optics allow us to use triangulation to calculate the three-dimensional positioning of the markers in space. This is the same principle used by geologists for land surveying.
Our technology is able to capture data at a rate of up to 220 frames per second. This results in long rows of numbers. If 37 markers with three spacial components each are all recorded over five minutes, this produces a set of roughly seven million values. (3 x 37 x 220 x 60 x 5 = ~7,000,000) From these data points, we can in turn extract further values, including distance covered, velocity and joint angles.
But what does motion capture footage have to do with linguistics at the ZAS? Every time we express our thoughts through speech, we activate multiple muscles, move our tongues and and make use of a whole range of articulators. This allows us to produce sound, syllables and entire words. Simultaneously, we produce facial expressions, nod or shake our heads to agree or disagree, shift our weight from one foot to another when we wish for a chance to speak, and gesticulate to underscore certain points. Furthermore, verbal communication tends to be embedded in parallel actions. We talk when we jog, while we shop and when we walk to school. Researchers from different departments here at the ZAS wish to study all of these movements (articulatory movements, facial movements, physical posture, gestures, walking/cycling) as well as their stability, frequency and coordinational aspects.