Unconstrained Face Recognition

Keywords: optics, sensors, eye detection, face recognition, feature descriptors, data fusion, evaluation
Spring 2008 - Present
teaser image

Description

The growing demand for highly accurate surveillance, intelligence, and forensics systems has propelled the unconstrained recognition problem (1:N matching; also known as "identification") to the forefront of computer vision research. Over the past decade, excellent progress has been made toward the constrained and unconstrained verification (1:1 matching) problems. For controlled face verification, 99.9% accuracy was achieved for the FRGC set. For uncontrolled face verification, 88.13% accuracy has been reported for the once very difficult LFW set. Verification is a fundamentally easier problem than recognition, as it only considers discrete pairs of samples for matching, with a claimed identity choosing a comparison class that is known to the matching system. Recognition is made more difficult by the need to identify an unknown class out of the set of known classes. In open set recognition, it is possible that the unknown class does not correspond to any known class, and thus should not match any class. Compounding these structural considerations is the overall environment of the unconstrained scenario, where any number of effects (pose, illumination, expression, sensor noise, weather, etc.) can impact accuracy. Recognition, in general, is a challenging problem with important consequences for security and forensics applications.

In this work, we have taken a systems level approach to unconstrained face recognition, exploring all facets of the problem from image acquisition to classification. Beginning with a comprehensive study of optics and sensors, we have isolated the key problems that inhibit good recognition with today’s technology. In order to compensate for such problems, new algorithms for image restoration, feature detection and classification have allowed us to achieve reasonable levels of recognition accuracy where many other approaches have failed. We have also placed considerable effort into developing novel approaches for algorithm evaluation, including a controlled semi-synthetic data acquisition platform that injects real environmental effects into standard data sets.

This work was supported by ONR SBIR Award No. N00014-11-C-0243, ONR STTR Award No. N000-14-07-M-0421, ONR MURI Award No. N00014-08-1-0638, Army SBIR Award No. W15P7T-12-C-A210, and NSF PFI Award No. 650251

Publications

  • "Detecting and Classifying Scars, Marks, and Tattoos Found in the Wild,"
    Brian Heflin, Walter J. Scheirer, Terrance E. Boult,
    Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS),
    September 2012.
  • "For Your Eyes Only,"
    Brian Heflin, Walter J. Scheirer, Terrance E. Boult,
    Proceedings of the IEEE Workshop on the Applications of Computer Vision (WACV),
    January 2012.
  • "Fusing with Context: a Bayesian Approach to Combining Descriptive Attributes,"
    Walter J. Scheirer, Neeraj Kumar, Karl Ricanek, Terrance E. Boult, Peter N. Belhumeur,
    Proceedings of the IEEE International Joint Conference on Biometrics (IJCB),
    October 2011.
  • "A Look at Eye Detection for Unconstrained Environments,"
    Brian Heflin, Walter J. Scheirer, Anderson Rocha, Terrance E. Boult,
    In Pattern Recognition, Machine Intelligence and Biometrics (Higher Education Press & Springer-Verlag),
    September 2011.
  • "Single Image Deblurring for a Real-Time Face Recognition System,"
    Brian Heflin, Brian Parks, Walter J. Scheirer, Terrance E. Boult,
    Proceedings of the 36th Annual Conference of the IEEE Industrial Electronics Society (IECON),
    November 2010.
  • "Correcting Rolling-Shutter Distortion for a Real-Time Face Recognition System,"
    Brian Heflin, Walter J. Scheirer, Terrance E. Boult,
    Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS),
    October 2010.
  • "Face System Evaluation Toolkit: Recognition is Harder than it Seems,"
    Vijay Iyer, Walter J. Scheirer, Terrance E. Boult,
    Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS),
    October 2010.
  • "FACE-GRAB: Face Recognition with General Region Assigned to Binary Operator,"
    Archana Sapkota, Brian Parks, Walter J. Scheirer, Terrance E. Boult,
    Proceedings of the IEEE Computer Society Workshop on Biometrics,
    June 2010.
  • "A Taxonomy of Face Models for System Evaluation,"
    Vijay Iyer, Shane Kirkbride, Brian Parks, Walter J. Scheirer, Terrance E. Boult,
    Proceedings of the IEEE Workshop on Analysis and Modeling of Faces and Gestures (AMFG),
    June 2010.
  • "Difficult Detection: A Comparison of Two Different Approaches to Eye Detection for Unconstrained Environments,"
    Walter J. Scheirer, Anderson Rocha, Brian Heflin, Terrance E. Boult,
    Proceedings of the IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS),
    October 2009.
  • "Long Range Facial Image Acquisition and Quality,"
    Terrance E. Boult, Walter J. Scheirer,
    In Handbook of Remote Biometrics: for Surveillance and Security (Springer-Verlag),
    August 2009.
  • "FAAD: Face at a Distance,"
    Terrance E. Boult, Walter J. Scheirer, Robert Woodworth,
    Proceedings of the SPIE Defense and Security Symposium,
    March 2008.