Walter J. Scheirer

walter.scheirer@nd.edu, @wjscheirer

Associate Professor in the Department of Computer Science and Engineering,
University of Notre Dame

Research Overview

My research is primarily focused around the problem of recognition, including the representations and algorithms supporting solutions to it. I am particularly interested in features and learning-based methods that apply to both vision and language, thus breaking away from the persistent compartmentalization of recognition tasks (something hinted at by David Marr over 30 years ago). This has led to some interesting, and often unconventional approaches that can be applied to a broad set of areas including computer vision, machine learning, human biometrics, and the digital humanities. Specifically, my work is looking at open set recognition, extreme value theory models for visual recognition, biologically-inspired learning algorithms, and stylometry.

Curriculum Vitae

PDF Format

Postdoctoral Fellows

Daniel Moreira

Graduate Students

Sophia Abraham

Sreya Banerjee

Zachariah Carmichael

Samuel Grieggs

Jin Huang

Jeffery Kinnison

Derek Prijatelj

Brandon RichardWebster

Bingyu Shen

Rosaura Vidal Mata (Co-Advised with Kevin Bowyer)

William Theisen

Teaching

Spring 2020: CSE 40567/60567 Computer Security

Fall 2019: CSE 40171 Artificial Intelligence

Spring 2019: CSE 40567/60567 Computer Security

Fall 2018: CSE 40171 Artificial Intelligence

Spring 2018: CSE 40567/44567/60567 Computer Security

Spring 2017: CSE 40567/60567 Computer Security

Spring 2016: CSE 40567/60567 Computer Security

Fall 2015: CSE 40537/60537 Biometrics

Books

Quantitative Intertextuality: Analyzing the Markers of Information Reuse (Springer-Nature) with Christopher W. Forstall

Extreme Value Theory-based Methods for Visual Recognition (Morgan & Claypool Publishers)

Recent Publications (see all)

"Bridging the Gap Between Computational Photography and Visual
Recognition,"
Rosaura G. VidalMata, Sreya Banerjee, Brandon RichardWebster, Michael Albright,
Pedro Davalos, Scott McCloskey, Ben Miller, Asong Tambo, Sushobhan Ghosh,
Sudarshan Nagesh, Ye Yuan, Yueyu Hu, Junru Wu, Wenhan Yang, Xiaoshuai Zhang,
Jiaying Liu, Zhangyang Wang, Hwann-Tzong Chen, Tzu-Wei Huang, Wen-Chi Chin,
Yi-Chun Li, Mahmoud Lababidi, Charles Otto, Walter J. Scheirer,
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI),

Accepted for Publicatiton in 2020.

[pdf]
[bibtex]
@misc{vidalmata2019bridging,
title = {Bridging the Gap Between Computational Photography and Visual
Recognition},
author = {Rosaura G. VidalMata and
Sreya Banerjee and
Brandon RichardWebster and
Michael Albright and
Pedro Davalos and
Scott McCloskey and
Ben Miller and
Asong Tambo and
Sushobhan Ghosh and
Sudarshan Nagesh and
Ye Yuan and
Yueyu Hu and
Junru Wu and
Wenhan Yang and
Xiaoshuai Zhang and
Jiaying Liu and
Zhangyang Wang and
Hwann-Tzong Chen and
Tzu-Wei Huang and
Wen-Chi Chin and
Yi-Chun Li and
Mahmoud Lababidi and
Charles Otto and
Walter J. Scheirer},
year = {2019},
eprint = {1901.09482},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}
"Backdooring Convolutional Neural Networks via Targeted Weight Perturbations,"
Jacob Dumford, Walter J. Scheirer,
Proceedings of the IAPR/IEEE International Joint Conference on Biometrics (IJCB),

September 2020.

[pdf] [talk]
[bibtex]
@inproceedings{Dumford20_IJCB,
author = {Jacob Dumford and
Walter J. Scheirer},
title = {Backdooring Convolutional Neural Networks via Targeted Weight
Perturbations},
booktitle = {Proceedings of the IAPR/IEEE International Joint Conference on
Biometrics (IJCB)},
year = {2020}
}
"The Next Generation of Human-Drone Partnerships: Co-Designing an Emergency
Response System,"
Ankit Agrawal, Sophia Abraham, Benjamin Burger, Chichi Christine, Luke Fraser,
John Hoeksema, Sara Hwang, Elizabeth Travnik, Shreya Kumar, Walter J. Scheirer,
Jane Cleland-Huang, Michael Vierhauser,
Ryan Bauer, Steve Cox,
Proceedings of the ACM CHI Conference on Human Factors in Computing Systems
(Honorable Mention Award),

April 2020.

[pdf]
[bibtex]
@misc{agrawal2020generation,
title={The Next Generation of Human-Drone Partnerships: Co-Designing an Emergency
Response System},
author={Ankit Agrawal and Sophia Abraham and Benjamin Burger and Chichi Christine
and Luke Fraser and John Hoeksema and Sara Hwang and Elizabeth Travnik and
Shreya Kumar and Walter Scheirer and Jane Cleland-Huang and Michael Vierhauser
and Ryan Bauer and Steve Cox},
year={2020},
eprint={2001.03849},
archivePrefix={arXiv},
primaryClass={cs.HC}
}
"An AI Early Warning System to Monitor Online Disinformation, Stop Violence,
and Protect Elections,"
Michael Yankoski, Tim Weninger, Walter J. Scheirer,
Bulletin of the Atomic Scientists,

March 2020.

[pdf]
[bibtex]
@article{yankoski2020ai,
title={An AI early warning system to monitor online disinformation, stop
violence, and protect elections},
author={Yankoski, Michael and Weninger, Tim and Scheirer, Walter},
journal={Bulletin of the Atomic Scientists},
month={March},
year={2020},
publisher={Taylor \& Francis}
}
"Self-Driving Vehicles: Key Technical Challenges and Progress Off the Road,"
Michael Milford, Samuel E. Anthony, Walter J. Scheirer,
IEEE Potentials,

January-February 2020.

[pdf]
[bibtex]
@article{MilfordPotentials2020,
author = {Michael Milford and
Samuel E. Anthony and
Walter J. Scheirer},
title = {Self-Driving Vehicles: Key Technical Challenges and Progress Off
the Road},
journal = {IEEE Potentials},
volume = {39},
number = {1},
month = {January-February},
year = {2020}
}
"PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition,"
Brandon RichardWebster, Samuel E. Anthony, Walter J. Scheirer,
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI),

September 2019.

[pdf]
[bibtex]
@article{RichardWebsterPsyPhy2019,
author = {Brandon RichardWebster and
Samuel E. Anthony and
Walter J. Scheirer},
title = {PsyPhy: {A} Psychophysics Driven Evaluation Framework for Visual
Recognition},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence
(T-PAMI)},
volume = {41},
number = {9},
month = {September},
year = {2019}
}
"A Neurobiological Evaluation Metric for Neural Network Model Search,"
Nathaniel Blanchard, Jeffery Kinnison, Brandon RichardWebster, Pouya Bashivan,
Walter J. Scheirer,
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR),

June 2019.

[pdf]
[bibtex]
@InProceedings{Blanchard19,
author = {Nathaniel Blanchard and
Jeffery Kinnison and
Brandon RichardWebster and
Pouya Bashivan and
Walter J. Scheirer},
title = {A Neurobiological Evaluation Metric for Neural Network Model Search},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition
(CVPR)},
year = {2019},
}
"Image Provenance Analysis at Scale,"
Daniel Moreira, Aparna Bharati, Joel Brogan, Allan Pinto, Michael Parowski,
Kevin Bowyer, Patrick Flynn, Anderson Rocha, Walter J. Scheirer,
IEEE Transactions on Image Processing (T-IP),

December 2018.

[pdf] [dataset] [code]
[bibtex]
@article{MoreiraProvenance2018,
author = {Daniel Moreira and
Aparna Bharati and
Joel Brogan and
Allan da Silva Pinto and
Michael Parowski and
Kevin W. Bowyer and
Patrick J. Flynn and
Anderson Rocha and
Walter J. Scheirer},
title = {Image Provenance Analysis at Scale},
journal = {IEEE Transactions on Image Processing (T-IP)},
volume = {27},
number = {12},
year = {2018}
}
"Visual Psychophysics for Making Face Recognition Algorithms More Explainable,"
Brandon RichardWebster, So Yon Kwon, Christopher Clarizio, Samuel E. Anthony,
Walter J. Scheirer,
Proceedings of the European Conference on Computer Vision (ECCV),

September 2018.

[pdf]
[bibtex]
@InProceedings{RichardWebesterFace2018,
author = {Brandon RichardWebster and
So Yon Kwon and
Christopher Clarizio and
Samuel E. Anthony and
Walter J. Scheirer},
title = {Visual Psychophysics for Making Face Recognition Algorithms More
Explainable},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2018}
}

New Research in arXiv / bioRxiv

"The Criminality From Face Illusion,"
Kevin Bowyer, Michael King, Walter J. Scheirer,
June 2020.

[pdf]
[bibtex]
@misc{bowyer2020criminality,
title={The Criminality From Face Illusion},
author={Kevin W. Bowyer and Michael King and Walter Scheirer},
year={2020},
eprint={2006.03895},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
"Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in
Untrimmed Sequences,"
Rosaura G. VidalMata, Walter J. Scheirer, Hildegard Kuehne,
January 2020.

[pdf]
[bibtex]
@misc{vidalmata2020joint,
title={Joint Visual-Temporal Embedding for Unsupervised Learning of Actions
in Untrimmed Sequences},
author={Rosaura G. VidalMata and Walter J. Scheirer and Hilde Kuehne},
year={2020},
eprint={2001.11122},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
"Automatic Discovery of Political Meme Genres with Diverse Appearances,"
William Theisen, Joel Brogan, Pamela Bilo Thomas, Daniel Moreira, Pascal Phoa,
Tim Weninger, Walter J. Scheirer,
January 2020.

[pdf]
[bibtex]
@misc{theisen2020automatic,
title={Automatic Discovery of Political Meme Genres with Diverse Appearances},
author={William Theisen and Joel Brogan and Pamela Bilo Thomas and
Daniel Moreira and Pascal Phoa and Tim Weninger and Walter Scheirer},
year={2020},
eprint={2001.06122},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
"Learning Transformation-Aware Embeddings for Image Forensics,"
Aparna Bharati, Daniel Moreira, Patrick Flynn, Anderson Rocha, Kevin Bowyer,
Walter J. Scheirer,
January 2020.

[pdf]
[bibtex]
@misc{bharati2020learning,
title={Learning Transformation-Aware Embeddings for Image Forensics},
author={Aparna Bharati and Daniel Moreira and Patrick Flynn and
Anderson Rocha and Kevin Bowyer and Walter Scheirer},
year={2020},
eprint={2001.04547},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
"Dynamic Spatial Verification for Large-Scale Object-Level Image Retrieval,"
Joel Brogan, Aparna Bharati, Daniel Moreira, Kevin Bowyer, Patrick Flynn,
Anderson Rocha, Walter J. Scheirer,
December 2019.

[pdf]
[bibtex]
@article{DBLP:journals/corr/abs-1903-10019,
author = {Joel Brogan and
Aparna Bharati and
Daniel Moreira and
Kevin W. Bowyer and
Patrick J. Flynn and
Anderson Rocha and
Walter J. Scheirer},
title = {Dynamic Spatial Verification for Large-Scale Object-Level
Image Retrieval},
journal = {CoRR},
volume = {abs/1903.10019},
year = {2019},
url = {http://arxiv.org/abs/1903.10019},
archivePrefix = {arXiv},
eprint = {1903.10019},
timestamp = {Mon, 01 Apr 2019 14:07:37 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1903-10019},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
"Report on UG^2+ Challenge Track 1: Assessing Algorithms to Improve Video
Object Detection and Classification from Unconstrained Mobility Platforms,"
Sreya Banerjee, Rosaura G. VidalMata, Zhangyang Wang, Walter J. Scheirer,
July 2019.

[pdf]
[bibtex]
@article{DBLP:journals/corr/abs-1907-11529,
author = {Sreya Banerjee and
Rosaura G. VidalMata and
Zhangyang Wang and
Walter J. Scheirer},
title = {Report on UG{\^{}}2+ Challenge Track 1: Assessing Algorithms to Improve
Video Object Detection and Classification from Unconstrained Mobility
Platforms},
journal = {CoRR},
volume = {abs/1907.11529},
year = {2019},
url = {http://arxiv.org/abs/1907.11529},
archivePrefix = {arXiv},
eprint = {1907.11529},
timestamp = {Thu, 01 Aug 2019 08:59:33 +0200},
biburl = {https://dblp.org/rec/bib/journals/corr/abs-1907-11529},
bibsource = {dblp computer science bibliography, https://dblp.org}
}
"Measuring Human Perception to Improve Handwritten Document Transcription,"
Samuel Grieggs, Bingyu Shen, Pei Li, Cana Short, Jiaqi Ma, Mihow McKenny,
Melody Wauke, Brian Price, Walter J. Scheirer,
April 2019.

[pdf]
[bibtex]
@article{DBLP:journals/corr/abs-1904-03734,
author = {Samuel Grieggs and
Bingyu Shen and
Pei Li and
Cana Short and
Jiaqi Ma and
Mihow McKenny and
Melody Wauke and
Brian Price and
Walter J. Scheirer},
title = {Measuring Human Perception to Improve Handwritten Document
Transcription},
journal = {CoRR},
volume = {abs/1904.03734},
year = {2019},
url = {http://arxiv.org/abs/1904.03734},
archivePrefix = {arXiv},
eprint = {1904.03734},
}

Recent Projects (see all)

Unconventional Approaches to Hacking Neural Networks

Spring 2018 - Present

Uncovering new vulnerabilities in neural network-based applications where security is a consideration

Keywords: machine learning, neural networks, face recognition, security, backdoors, exploits

Hardware-Aware Distributed Hyperparameter Optimization

Spring 2017 - Present

Intelligent mapping of hardware resources to model search tasks

Keywords: machine learning, hyperparameters, optimization, distributed computing, connectomics

Computational Models of Zebrafish Sensory Integration

Fall 2016 - Present

Modeling neural circuits to explain fish behavior and apply to generalized recognition problems in AI

Keywords: computational neuroscience, zebrafish, sensory integration, data fusion, extreme value theory

Image Restoration and Enhancement for Visual Recognition

Fall 2016 - Present

Work to bridge the gap between computational photography and visual recognition

Keywords: deblurring, denoising, super-resolution, deep learning, object recognition, video

Face Synthesis

Fall 2015 - Present

Fast and accurate photo-quality face synthesis for use in face recognition work

Keywords: face synthesis, face recognition, face analysis, computer graphics, data augmentation

Learning-free Image Segmentation

Fall 2015 - Present

Image segmentation for settings where machine learning-based approaches are slow and do not generalize

Keywords: image filtering, volumetric segmentation, connectomics, medical imaging, iris recognition

Tools for Neuroscience

Fall 2015 - Present

Techniques to help neuroscientists understand neuroanatomy and function

Keywords: connectomics, electron microscopy, two-photon imaging, x-ray tomography, 3D reconstruction

Brain-informed Machine Learning

Fall 2014 - Present

Applying measurements of neural activity to regularize machine learning models

Keywords: fMRI, psychophysics, classification, deep learning, neural-weights, object recognition

Visual Place Recognition

Spring 2013 - Present

Condition invariant visual place recognition for robot navigation inspired by vision science

Keywords: robotics, navigation, place recognition, whole image matching, patch matching, calibration

Good Recognition is Non-metric

Fall 2012 - Spring 2014

A new look at a fundamental question in computer vision: is recognition metric?

Keywords: machine learning, metric learning, recognition, face recognition, object recognition

Psychophysics for Computer Vision

Fall 2012 - Present

Measuring exemplar-by-exemplar difﬁculty and the pattern of errors of humans for supervised learning

Keywords: machine learning, psychology, citizen science, psychophysics, face detection, attributes

Open Set Recognition

Spring 2011 - Present

Theory and algorithms that address the difficult problem of training without complete class knowledge

Keywords: machine learning, object recognition, face recognition, support vector machines, 1-vs-set machine

Extreme Value Theory for Visual Recognition

Spring 2008 - Present

The theory and practice of recognition score analysis for prediction and fusion

Keywords: calibration, meta-recognition, score analysis, object recognition, face recognition, attributes

Language & Literature

Spring 2009 - Present

Machine learning and related statistical methods to improve the process by which intertextuality is studied

Keywords: computational linguistics, intertextuality, sound, stylistics, classics, authorship attribution

Digital Image and Video Forensics

Spring 2008 - Present

Approaches to steganalysis, forgery detection and sensor fingerprinting

Keywords: forensics, forgery detection, data hiding, source identification, sensor fingerprinting

Recent Talks (see all)

Invited talk at the WACV 2020 Workshop Neural Architecture Search for Computer Vision in the Wild: Representational Dissimilarity Analysis as a Tool for Neural Network Model Search
Talk at the 2019 Beijing Forum Panel on Humanity and Legal Governance in the Digital Era: New Perspectives on Digital Media Integrity
Keynote talk at the 18th International Conference of the Biometrics Special Interest Group (BIOSIG 2019): The Limits and Potentials of Deep Learning for Facial Analysis
Talk at the DEF CON 27 AI Village: Backdooring Convolutional Neural Networks with Targeted Weight Perturbations
Invited talk at the CVPR 2018 Workshop "Real-World Challenges and New Benchmarks for Deep Learning in Robotic Vision":
A Psychophysics Driven Evaluation Framework for Visual Recognition
Invited talk at Amazon: Using Human Behavior and Brain Activity to Guide Machine Learning
Invited talk at Cultural Analytics 2017: Quantitative Intertextuality: Analyzing the Markers of Information Reuse
Invited talk at Argonne National Laboratory: Scalable Strategies for Image Analysis in Neuroscience
CVPR 2016 Tutorial: Statistical Methods for Open Set Recognition
Invited talk at the RSS 2016 Workshop "Are the Sceptics Right? Limits and Potentials of Deep Learning in Robotics":
The Impact of the Open Set Recognition Problem on Deep Learning (Video)

Activities

CVPR 2020 was held virtually, June 14th - 19th
WACV 2020 was held in Snowmass Village, CO, March 2nd - 5th
CVPR 2019 was held in Long Beach, California, June 15th - 21st
The UG^2+ Challenge prize challenge workshop on image restoration and enhancement was held at CVPR 2019
The Interactive and Adaptive Learning in an Open World workshop was co-located with ECCV 2018
The UG^2 Challenge prize challenge workshop on image restoration and enhancement was held at CVPR 2018
Join the PAMI-TC (now on LinkedIn) and the Computer Vision Foundation