3D facial tracking from corrupted movie sequences

Siome Goldenstein, Christian Vogler, Dimitri Metaxas

Research output: Contribution to journalConference article

12 Citations (Scopus)

Abstract

In this paper we perform 3D face tracking on corrupted video sequences. We use a deformable model, combined with a predictive filter, to recover both the rigid transformations and the values of the parameters that describe the evolution of the facial expressions over time. To be robust, predictive filters need a good observation of the system's state. We describe a new method to measure, at each moment in time, the correct distribution of an observation of the parameters of a high-dimensional deformable model. This method is based on bounding the confidence regions of the 2D image displacements with affine forms, and propagating them into parameter space. Using Lindeberg's theorem, we measure a good Gaussian approximation of the parameters in a manner that avoids many of the traditional assumptions about the observations' distributions. We demonstrate in experiments on sequences with compression artifacts, and poor-quality video sequences of Lauren Bacall and Humphrey Bogart from the 1950s, that, without any learning involved, our method is sufficiently robust to extract information from degraded image sequences. In addition, we provide ground truth validation.

Original languageEnglish (US)
JournalProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume1
StatePublished - Oct 19 2004
EventProceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004 - Washington, DC, United States
Duration: Jun 27 2004Jul 2 2004

Fingerprint

Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition

Keywords

  • 3D face tracking
  • Deformable models
  • Kalman filter
  • Parametric estimation
  • Particle Filter

Cite this

@article{bd26501605894fb795725ca3f31c9241,
title = "3D facial tracking from corrupted movie sequences",
abstract = "In this paper we perform 3D face tracking on corrupted video sequences. We use a deformable model, combined with a predictive filter, to recover both the rigid transformations and the values of the parameters that describe the evolution of the facial expressions over time. To be robust, predictive filters need a good observation of the system's state. We describe a new method to measure, at each moment in time, the correct distribution of an observation of the parameters of a high-dimensional deformable model. This method is based on bounding the confidence regions of the 2D image displacements with affine forms, and propagating them into parameter space. Using Lindeberg's theorem, we measure a good Gaussian approximation of the parameters in a manner that avoids many of the traditional assumptions about the observations' distributions. We demonstrate in experiments on sequences with compression artifacts, and poor-quality video sequences of Lauren Bacall and Humphrey Bogart from the 1950s, that, without any learning involved, our method is sufficiently robust to extract information from degraded image sequences. In addition, we provide ground truth validation.",
keywords = "3D face tracking, Deformable models, Kalman filter, Parametric estimation, Particle Filter",
author = "Siome Goldenstein and Christian Vogler and Dimitri Metaxas",
year = "2004",
month = "10",
day = "19",
language = "English (US)",
volume = "1",
journal = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",
issn = "1063-6919",
publisher = "IEEE Computer Society",

}

3D facial tracking from corrupted movie sequences. / Goldenstein, Siome; Vogler, Christian; Metaxas, Dimitri.

In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 19.10.2004.

Research output: Contribution to journalConference article

TY - JOUR

T1 - 3D facial tracking from corrupted movie sequences

AU - Goldenstein, Siome

AU - Vogler, Christian

AU - Metaxas, Dimitri

PY - 2004/10/19

Y1 - 2004/10/19

N2 - In this paper we perform 3D face tracking on corrupted video sequences. We use a deformable model, combined with a predictive filter, to recover both the rigid transformations and the values of the parameters that describe the evolution of the facial expressions over time. To be robust, predictive filters need a good observation of the system's state. We describe a new method to measure, at each moment in time, the correct distribution of an observation of the parameters of a high-dimensional deformable model. This method is based on bounding the confidence regions of the 2D image displacements with affine forms, and propagating them into parameter space. Using Lindeberg's theorem, we measure a good Gaussian approximation of the parameters in a manner that avoids many of the traditional assumptions about the observations' distributions. We demonstrate in experiments on sequences with compression artifacts, and poor-quality video sequences of Lauren Bacall and Humphrey Bogart from the 1950s, that, without any learning involved, our method is sufficiently robust to extract information from degraded image sequences. In addition, we provide ground truth validation.

AB - In this paper we perform 3D face tracking on corrupted video sequences. We use a deformable model, combined with a predictive filter, to recover both the rigid transformations and the values of the parameters that describe the evolution of the facial expressions over time. To be robust, predictive filters need a good observation of the system's state. We describe a new method to measure, at each moment in time, the correct distribution of an observation of the parameters of a high-dimensional deformable model. This method is based on bounding the confidence regions of the 2D image displacements with affine forms, and propagating them into parameter space. Using Lindeberg's theorem, we measure a good Gaussian approximation of the parameters in a manner that avoids many of the traditional assumptions about the observations' distributions. We demonstrate in experiments on sequences with compression artifacts, and poor-quality video sequences of Lauren Bacall and Humphrey Bogart from the 1950s, that, without any learning involved, our method is sufficiently robust to extract information from degraded image sequences. In addition, we provide ground truth validation.

KW - 3D face tracking

KW - Deformable models

KW - Kalman filter

KW - Parametric estimation

KW - Particle Filter

UR - http://www.scopus.com/inward/record.url?scp=5044222276&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=5044222276&partnerID=8YFLogxK

M3 - Conference article

VL - 1

JO - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

JF - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SN - 1063-6919

ER -