Now there is a computer program that allows you to control someone else’s face – Quartz


In the prophetic 1997 film “Face / Off”, John Travolta believes that the only way to avoid going back to prison was to surgically replace his face with that of Nicolas Cage. While medical science hasn’t quite gotten to the point where it’s not completely laughable just yet, new technology from Stanford University is bringing us closer.

Researchers have discovered how to make one person’s face mimic another’s facial expressions, in real-time video. The method, announced in an article (pdf) that will appear in a special edition of the scientific journal ACM Transactions on Graphics later this year, uses a regular computer, special cameras, and seemingly magical new software.

The research team includes computer scientists from Stanford, the Max Planck Institute and the University of Erlangen-Nuremberg in Germany.

Their system requires a bit of configuration: a pair of cameras must calibrate themselves with each new face, then render it in digital 3D. Then, the program tracks the facial expressions of the two subjects using cameras capable of detecting the depth, texture, shape and location of the face, and maps the movements of important facial features like the nose, mouth, and face. the eyes, from one person’s face to the other’s avatar. The end result: You can seem to control a friend’s face with your own.

(To make up for the fact that not everyone’s mouths are the same size, the program puts together a weirdly fake set of perfect teeth to fill in the gaps.)

Real-time expression transfer for facial reconstruction / Thies, Zollhöfer, Nießner et al.

It’s like Photoshop, but for video.

Matthias Niessner, one of the researchers on the project, told Quartz that the team’s main motivation was to create something that could facilitate multilingual video conferencing like Skype.

YouTube / Matthias Niessner

David Bowie might be interested in seeing this.

In the future, interpreters could translate someone speaking in real time, and the end user would simply see the person they are watching speaking to them in their own language.

In their research paper, the team said they believe this technology could pave the way for photo-realistic avatars in virtual reality environments.

Niessner added that the team is also interested in applying this technology to films, dubbing them for foreign audiences. “Most importantly though: it’s a ton of fun playing with the system,” Niessner said.


Gordon K. Morehouse