Walter J. Scheirer

walter.scheirer@nd.edu, @wjscheirer

Assistant Professor in the Department of Computer Science and Engineering,
University of Notre Dame

Research Overview

My research is primarily focused around the problem of recognition, including the representations and algorithms supporting solutions to it. I am particularly interested in features and learning-based methods that apply to both vision and language, thus breaking away from the persistent compartmentalization of recognition tasks (something hinted at by David Marr over 30 years ago). This has led to some interesting, and often unconventional approaches that can be applied to a broad set of areas including computer vision, machine learning, human biometrics, and the digital humanities. Specifically, my work is looking at open set recognition, extreme value theory models for visual recognition, biologically-inspired learning algorithms, and stylometry.

New Research in arXiv / bioRxiv

1. "Report on UG^2+ Challenge Track 1: Assessing Algorithms to Improve Video
   Object Detection and Classification from Unconstrained Mobility Platforms,"

   Sreya Banerjee, Rosaura G. VidalMata, Zhangyang Wang, Walter J. Scheirer,

   July 2019.

   [pdf]

   [bibtex]

   @article{DBLP:journals/corr/abs-1907-11529,
   author = {Sreya Banerjee and
   Rosaura G. VidalMata and
   Zhangyang Wang and
   Walter J. Scheirer},
   title = {Report on UG{\^{}}2+ Challenge Track 1: Assessing Algorithms to Improve
   Video Object Detection and Classification from Unconstrained Mobility
   Platforms},
   journal = {CoRR},
   volume = {abs/1907.11529},
   year = {2019},
   url = {http://arxiv.org/abs/1907.11529},
   archivePrefix = {arXiv},
   eprint = {1907.11529},
   timestamp = {Thu, 01 Aug 2019 08:59:33 +0200},
   biburl = {https://dblp.org/rec/bib/journals/corr/abs-1907-11529},
   bibsource = {dblp computer science bibliography, https://dblp.org}
   }
   
2. "Measuring Human Perception to Improve Handwritten Document Transcription,"

   Samuel Grieggs, Bingyu Shen, Pei Li, Cana Short, Jiaqi Ma, Mihow McKenny,
   Melody Wauke, Brian Price, Walter J. Scheirer,

   April 2019.

   [pdf]

   [bibtex]

   @article{DBLP:journals/corr/abs-1904-03734,
   author = {Samuel Grieggs and
   Bingyu Shen and
   Pei Li and
   Cana Short and
   Jiaqi Ma and
   Mihow McKenny and
   Melody Wauke and
   Brian Price and
   Walter J. Scheirer},
   title = {Measuring Human Perception to Improve Handwritten Document Transcription},
   journal = {CoRR},
   volume = {abs/1904.03734},
   year = {2019},
   url = {http://arxiv.org/abs/1904.03734},
   archivePrefix = {arXiv},
   eprint = {1904.03734},
   }
   
3. "Needle in a Haystack: A Framework for Seeking Small Objects in Big Datasets,"

   Joel Brogan, Aparna Bharati, Daniel Moreira, Kevin Bowyer, Patrick Flynn,
   Anderson Rocha, Walter J. Scheirer,

   March 2019.

   [pdf]

   [bibtex]

   @article{DBLP:journals/corr/abs-1903-10019,
   author = {Joel Brogan and
   Aparna Bharati and
   Daniel Moreira and
   Kevin W. Bowyer and
   Patrick J. Flynn and
   Anderson Rocha and
   Walter J. Scheirer},
   title = {Needle in a Haystack: {A} Framework for Seeking Small Objects in Big Datasets},
   journal = {CoRR},
   volume = {abs/1903.10019},
   year = {2019},
   url = {http://arxiv.org/abs/1903.10019},
   archivePrefix = {arXiv},
   eprint = {1903.10019},
   timestamp = {Mon, 01 Apr 2019 14:07:37 +0200},
   biburl = {https://dblp.org/rec/bib/journals/corr/abs-1903-10019},
   bibsource = {dblp computer science bibliography, https://dblp.org}
   }
   
4. "Bridging the Gap Between Computational Photography and Visual Recognition,"

   Rosaura G. VidalMata, Sreya Banerjee, Brandon RichardWebster, Michael Albright,
   Pedro Davalos, Scott McCloskey, Ben Miller, Asong Tambo, Sushobhan Ghosh,
   Sudarshan Nagesh, Ye Yuan, Yueyu Hu, Junru Wu, Wenhan Yang, Xiaoshuai Zhang,
   Jiaying Liu, Zhangyang Wang, Hwann-Tzong Chen, Tzu-Wei Huang, Wen-Chi Chin,
   Yi-Chun Li, Mahmoud Lababidi, Charles Otto, Walter J. Scheirer,

   January 2019.

   [pdf]

   [bibtex]

   @misc{vidalmata2019bridging,
   title = {Bridging the Gap Between Computational Photography and Visual Recognition},
   author = {Rosaura G. VidalMata and
   Sreya Banerjee and
   Brandon RichardWebster and
   Michael Albright and
   Pedro Davalos and
   Scott McCloskey and
   Ben Miller and
   Asong Tambo and
   Sushobhan Ghosh and
   Sudarshan Nagesh and
   Ye Yuan and
   Yueyu Hu and
   Junru Wu and
   Wenhan Yang and
   Xiaoshuai Zhang and
   Jiaying Liu and
   Zhangyang Wang and
   Hwann-Tzong Chen and
   Tzu-Wei Huang and
   Wen-Chi Chin and
   Yi-Chun Li and
   Mahmoud Lababidi and
   Charles Otto and
   Walter J. Scheirer},
   year = {2019},
   eprint = {1901.09482},
   archivePrefix = {arXiv},
   primaryClass = {cs.CV}
   }
   
5. "Backdooring Convolutional Neural Networks via Targeted Weight Perturbations,"

   Jacob Dumford, Walter J. Scheirer,

   December 2018.

   [pdf]

   [bibtex]

   @article{DBLP:journals/corr/Dumford18,
   author = {Jacob Dumford and
   Walter J. Scheirer},
   title = {Backdooring Convolutional Neural Networks via Targeted Weight Perturbations},
   journal = {CoRR},
   volume = {abs/1812.03128},
   year = {2018},
   url = {https://arxiv.org/abs/1812.03128}
   }
   
6. "On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs,"

   Sandipan Banerjee, Walter J. Scheirer, Kevin Bowyer, Patrick Flynn,

   November 2018.

   [pdf]

   [bibtex]

   @article{DBLP:journals/corr/abs-1811-07104,
   author = {Sandipan Banerjee and
   Walter J. Scheirer and
   Kevin W. Bowyer and
   Patrick J. Flynn},
   title = {On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs},
   journal = {CoRR},
   volume = {abs/1811.07104},
   year = {2018},
   url = {http://arxiv.org/abs/1811.07104},
   archivePrefix = {arXiv},
   eprint = {1811.07104},
   timestamp = {Sun, 25 Nov 2018 18:57:12 +0100},
   biburl = {https://dblp.org/rec/bib/journals/corr/abs-1811-07104},
   bibsource = {dblp computer science bibliography, https://dblp.org}
   }
   

Recent Projects (see all)

Hardware-Aware Distributed Hyperparameter Optimization

Spring 2017 - Present

Intelligent mapping of hardware resources to model search tasks

Keywords: machine learning, hyperparameters, optimization, distributed computing, connectomics

Computational Models of Zebrafish Sensory Integration

Fall 2016 - Present

Modeling neural circuits to explain fish behavior and apply to generalized recognition problems in AI

Keywords: computational neuroscience, zebrafish, sensory integration, data fusion, extreme value theory

Image Restoration and Enhancement for Visual Recognition

Fall 2016 - Present

Work to bridge the gap between computational photography and visual recognition

Keywords: deblurring, denoising, super-resolution, deep learning, object recognition, video

Face Synthesis

Fall 2015 - Present

Fast and accurate photo-quality face synthesis for use in face recognition work

Keywords: face synthesis, face recognition, face analysis, computer graphics, data augmentation

Learning-free Image Segmentation

Fall 2015 - Present

Image segmentation for settings where machine learning-based approaches are slow and do not generalize

Keywords: image filtering, volumetric segmentation, connectomics, medical imaging, iris recognition

Tools for Neuroscience

Fall 2015 - Present

Techniques to help neuroscientists understand neuroanatomy and function

Keywords: connectomics, electron microscopy, two-photon imaging, x-ray tomography, 3D reconstruction

Brain-informed Machine Learning

Fall 2014 - Present

Applying measurements of neural activity to regularize machine learning models

Keywords: fMRI, psychophysics, classification, deep learning, neural-weights, object recognition

Visual Place Recognition

Spring 2013 - Present

Condition invariant visual place recognition for robot navigation inspired by vision science

Keywords: robotics, navigation, place recognition, whole image matching, patch matching, calibration

Good Recognition is Non-metric

Fall 2012 - Spring 2014

A new look at a fundamental question in computer vision: is recognition metric?

Keywords: machine learning, metric learning, recognition, face recognition, object recognition

Psychophysics for Computer Vision

Fall 2012 - Present

Measuring exemplar-by-exemplar difficulty and the pattern of errors of humans for supervised learning

Keywords: machine learning, psychology, citizen science, psychophysics, face detection, attributes

Open Set Recognition

Spring 2011 - Present

Theory and algorithms that address the difficult problem of training without complete class knowledge

Keywords: machine learning, object recognition, face recognition, support vector machines, 1-vs-set machine

Extreme Value Theory for Visual Recognition

Spring 2008 - Present

The theory and practice of recognition score analysis for prediction and fusion

Keywords: calibration, meta-recognition, score analysis, object recognition, face recognition, attributes

Language & Literature

Spring 2009 - Present

Machine learning and related statistical methods to improve the process by which intertextuality is studied

Keywords: computational linguistics, intertextuality, sound, stylistics, classics, authorship attribution

Digital Image and Video Forensics

Spring 2008 - Present

Approaches to steganalysis, forgery detection and sensor fingerprinting

Keywords: forensics, forgery detection, data hiding, source identification, sensor fingerprinting

Recent Talks (see all)

• Keynote talk at the 18th International Conference of the Biometrics Special Interest Group (BIOSIG 2019): The Limits and Potentials of Deep Learning for Facial Analysis
• Talk at the DEF CON 27 AI Village: Backdooring Convolutional Neural Networks with Targeted Weight Perturbations
• Invited talk at the CVPR 2018 Workshop "Real-World Challenges and New Benchmarks for Deep Learning in Robotic Vision":
        A Psychophysics Driven Evaluation Framework for Visual Recognition
• Invited talk at Amazon: Using Human Behavior and Brain Activity to Guide Machine Learning
• Invited talk at Cultural Analytics 2017: Quantitative Intertextuality: Analyzing the Markers of Information Reuse
• Invited talk at Argonne National Laboratory: Scalable Strategies for Image Analysis in Neuroscience
• CVPR 2016 Tutorial: Statistical Methods for Open Set Recognition
• Invited talk at the RSS 2016 Workshop "Are the Sceptics Right? Limits and Potentials of Deep Learning in Robotics":
        The Impact of the Open Set Recognition Problem on Deep Learning (Video)

Activities

• CVPR 2020 will be held in Seattle, Washington, June 14th - 19th
• WACV 2020 will be held in Snowmass Village, CO, March 2nd - 5th
• CVPR 2019 was held in Long Beach, California, June 15th - 21st
• The UG^2+ Challenge prize challenge workshop on image restoration and enhancement was held at CVPR 2019
• The Interactive and Adaptive Learning in an Open World workshop was co-located with ECCV 2018
• The UG^2 Challenge prize challenge workshop on image restoration and enhancement was held at CVPR 2018
• Join the PAMI-TC (now on LinkedIn) and the Computer Vision Foundation