walter.scheirer@nd.edu, @wjscheirer, @wjscheirer.bsky.social

Dennis O. Doughty Collegiate Associate Professor in the Department of Computer Science and Engineering,
University of Notre Dame

Research Overview

My technical research is primarily focused around the problem of recognition, including the representations and algorithms supporting solutions to it. I am particularly interested in features and learning-based methods that apply to both vision and language, thus breaking away from the persistent compartmentalization of recognition tasks (something hinted at by David Marr over 30 years ago). This has led to some intriguing and often unconventional approaches that can be applied to a broad set of areas within artificial intelligence including computer vision, machine learning, and human biometrics. Specifically, my work is looking at open set recognition, extreme value theory models for visual recognition, and biologically-inspired learning algorithms. I'm also a cultural critic and historian, commenting on the social context of emerging technologies from the realistic perspective of a computer scientist.


Walter J. Scheirer

Curriculum Vitae

PDF Format

Postdoctoral Fellows

Steve Cruz (Co-Advised with Jonathan Hauenstein)

Graduate Students

Sophia Abraham

Zachariah Carmichael

Louisa Conwill

Justin Dulay

Jin Huang

Derek Prijatelj

Arturo Russell (Co-Advised with Jane Cleland-Huang)

Abigail Swenor

William Theisen

Teaching

Spring 2024: CSE 20176 The Archeology of Hacking

Fall 2023: CSE 40175 Ethical & Professional Issues

Spring 2023: CSE 20176 The Archeology of Hacking

Fall 2022: CSE 40175 Ethical & Professional Issues

Spring 2022: CSE 40175 Ethical & Professional Issues

Fall 2021: CSE 40175 Ethical & Professional Issues

Spring 2020: CSE 40567/60567 Computer Security

Fall 2019: CSE 40171 Artificial Intelligence

Spring 2019: CSE 40567/60567 Computer Security

Fall 2018: CSE 40171 Artificial Intelligence

Spring 2018: CSE 40567/44567/60567 Computer Security

Spring 2017: CSE 40567/60567 Computer Security

Spring 2016: CSE 40567/60567 Computer Security

Fall 2015: CSE 40537/60537 Biometrics

Books

A History of Fake Things on the Internet (Stanford University Press)

A Unifying Framework for Formal Theories of Novelty (Springer-Nature) edited with Terry Boult

Quantitative Intertextuality: Analyzing the Markers of Information Reuse (Springer-Nature) with Christopher W. Forstall

Extreme Value Theory-based Methods for Visual Recognition (Morgan & Claypool Publishers)

Recent Publications (see all)

  1. "Automated Transcription of Gə'əz Manuscripts Using Deep Learning,"
    Samuel Grieggs, Jessica Lockhart, Alexandra Atiya, Suzanne Akbari,
    Eyob Derillo, Jarod Jacobs, Christine Kwon, Michael Gervers, Steve Delamarter,
    Alexandra Gillespie, Walter J. Scheirer,
    Digital Humanities Quarterly,
    August 2023.
    [pdf]
    [bibtex]
    @article{Grieggs_DHQ2023,
    author = {Samuel Grieggs and
    Jessica Lockhart and
    Alexandra Atiya and
    Gelila Tilahun and
    Suzanne Akbari and
    Eyob Derillo and
    Jarod Jacobs
    Christine Kwon
    Michael Gervers
    Steve Delamarter
    Walter J. Scheirer},
    title = {Automated Transcription of Gə'əz Manuscripts Using Deep Learning},
    journal = {Digital Humanities Quarterly},
    volume = {17},
    number = {3},
    month = {August},
    year = {2023}
    }
  2. "Psychophysical-Score: A Behavioral Measure for Assessing the Biological
    Plausibility of Visual Recognition Models,"
    Brandon RichardWebster, Justin Dulay, Anthony DiFalco, Elisabetta Caldesi,
    Walter Scheirer,
    CogSci 2023,
    July 2023.
    [pdf]
    [bibtex]
    @inProceedings{richardwebster2023psychophysicalscore,
    title={Psychophysical-Score: A Behavioral Measure for Assessing the Biological
    Plausibility of Visual Recognition Models}, author={Brandon RichardWebster and
    Justin Dulay and
    Anthony DiFalco and
    Elisabetta Caldesi and
    Walter J. Scheirer},
    booktitle={CogSci},
    year={2023},
    }
  3. "Measuring Human Perception to Improve Open Set Recognition,"
    Jin Huang, Derek Prijatelj, Justin Dulay, Walter Scheirer,
    IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI),
    Accepted for Publication in April 2023.
    [pdf] [supp. material] [code] [data]
    [bibtex]
    @misc{Huang2022,
    doi = {10.48550/ARXIV.2209.03519},
    url = {https://arxiv.org/abs/2209.03519},
    author = {Huang, Jin and
    Prijatelj, Derek and
    Dulay, Justin and
    Scheirer, Walter},
    title = {Measuring Human Perception to Improve Open Set Recognition},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution Share Alike 4.0 International}
    }
  4. "Unfooling Perturbation-Based Post Hoc Explainers,"
    Zachariah Carmichael, Walter J. Scheirer,
    Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2023),
    February 2023.
    [pdf]
    [bibtex]
    @inproceedings{Carmichael2023,
    url = {https://arxiv.org/abs/2205.14772},
    author = {Carmichael, Zachariah and
    Scheirer, Walter},
    title = {Unfooling Perturbation-Based Post Hoc Explainers},
    booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence
    (AAAI 2023)},
    year = {2023}
    }
  5. "Motif Mining: Finding and Summarizing Remixed Image Content,"
    William Theisen, Daniel Gonzalez, Zachariah Carmichael, Daniel Moreira,
    Tim Weninger, Walter Scheirer,
    Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision
    (WACV),
    January 2023.
    [pdf]
    [bibtex]
    @inproceedings{Theisen_2023,
    author = {Theisen, William and
    Cedre, Daniel Gonzalez and
    Carmichael, Zachariah and
    Moreira, Daniel and
    Weninger, Tim and
    Scheirer, Walter},
    title = {Motif Mining: Finding and Summarizing Remixed Image Content},
    publisher = {arXiv},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on
    Applications of Computer Vision (WACV)},
    year = {2023},
    }
  6. "SILA: A System For Scientific Image Analysis,"
    Daniel Moreira, Joao P. Cardenuto, Ruiting Shao, Sriram Baireddy, Davide Cozzolino,
    Diego Gragnaniello    , Wael Abd-Almageed, Paolo Bestagini, Stefano Tubaro,
    Anderson Rocha, Walter J. Scheirer, Luisa Verdoliva, Edward J. Delp,
    Scientific Reports,
    October 2022.
    [pdf]
    [bibtex]
    @article{moreira2022sila,
    title={SILA: a system for scientific image analysis},
    author={Moreira, Daniel and
    Cardenuto, Jo{\~a}o Phillipe and
    Shao, Ruiting and
    Baireddy, Sriram and
    Cozzolino, Davide and
    Gragnaniello, Diego and
    Abd-Almageed, Wael and
    Bestagini, Paolo and
    Tubaro, Stefano and
    Rocha, Anderson and
    Scheirer, Walter and
    Verdoliva, Luisa and
    Delp, Edward},
    journal={Scientific Reports},
    volume={12},
    number={1},
    pages={1--15},
    year={2022},
    publisher={Nature Publishing Group}
    }
  7. "Measuring Human Perception to Improve Handwritten Document Transcription,"
    Samuel Grieggs, Bingyu Shen, Greta Rauch, Pei Li, Jiaqi Ma, David Chiang,
    Brian Price, Walter J. Scheirer,
    IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI),
    October 2022.
    [pdf]
    [bibtex]
    @article{grieggs2021measuring,
    title={Measuring human perception to improve handwritten document transcription},
    author={Grieggs, Samuel and
    Shen, Bingyu and
    Rauch, Greta and
    Li, Pei and
    Ma, Jiaqi and
    Chiang, David and
    Price, Brian andi
    Scheirer, Walter J},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
    volume={44},
    number={10},
    pages={6594--6601},
    year={2021},
    publisher={IEEE}
    }
  8. "Bridging the Gap Between Computational Photography and Visual
    Recognition,"
    Rosaura G. VidalMata, Sreya Banerjee, Brandon RichardWebster, Michael Albright,
    Pedro Davalos, Scott McCloskey, Ben Miller, Asong Tambo, Sushobhan Ghosh,
    Sudarshan Nagesh, Ye Yuan, Yueyu Hu, Junru Wu, Wenhan Yang, Xiaoshuai Zhang,
    Jiaying Liu, Zhangyang Wang, Hwann-Tzong Chen, Tzu-Wei Huang, Wen-Chi Chin,
    Yi-Chun Li, Mahmoud Lababidi, Charles Otto, Walter J. Scheirer,
    IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI),
    December 2021.
    [pdf]
    [bibtex]
    @article{vidalmata2021bridging,
    title = {Bridging the Gap Between Computational Photography and Visual
    Recognition},
    author = {Rosaura G. VidalMata and
    Sreya Banerjee and
    Brandon RichardWebster and
    Michael Albright and
    Pedro Davalos and
    Scott McCloskey and
    Ben Miller and
    Asong Tambo and
    Sushobhan Ghosh and
    Sudarshan Nagesh and
    Ye Yuan and
    Yueyu Hu and
    Junru Wu and
    Wenhan Yang and
    Xiaoshuai Zhang and
    Jiaying Liu and
    Zhangyang Wang and
    Hwann-Tzong Chen and
    Tzu-Wei Huang and
    Wen-Chi Chin and
    Yi-Chun Li and
    Mahmoud Lababidi and
    Charles Otto and
    Walter J. Scheirer},
    journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence
    (T-PAMI)},
    volume = {43},
    number = {12},
    month = {December},
    year = {2021}
    }
  9. "Handwriting Recognition with Novelty,"
    Derek Prijatelj, Samuel Grieggs, Futoshi Yumoto, Eric Robertson, Walter J. Scheirer,
    International Conference on Document Analysis and Recognition (ICDAR),
    September 2021.
    [pdf]
    [bibtex]
    @inproceedings{Prijatelj_ICDAR2021,
    author = {Derek Prijatelj and
    Samuel Grieggs and
    Futoshi Yumoto and
    Eric Robertson and
    Walter J. Scheirer},
    title = {Handwriting Recognition with Novelty},
    booktitle = {International Conference on Document Analysis and Recognition
    (ICDAR},
    year = {2021}
    }
  10. "Towards a Unifying Framework for Formal Theories of Novelty,"
    Terrance Boult, Przemyslaw Grabowicz, Derek Prijatelj, Roni Stern, Lawrence Holder,
    Joshua Alspector, Mohsen Jafarzadeh, Touqeer Ahmad, Akshay Dhamija, Chunchun Li,
    Steve Cruz, Abhinav Shrivastava, Carl Vondrick, Walter J. Scheirer,
    AAAI Conference on Artificial Intelligence (AAAI 2021), Senior Member Track,
    February 2021.
    [pdf]
    [bibtex]
    @inproceedings{Boult21,
    author = {Terrance Boult and
    Przemyslaw Grabowicz and
    Derek Prijatelj and
    Lawrence Holder and
    Joshua Alspector and
    Mohsen Jafarzadeh and
    Touqeer Ahmad and
    Akshay Dhamija and
    Chunchun Li and
    Steve Cruz and
    Abhinav Shrivastava and
    Carl Vondrick and
    Walter J. Scheirer},
    title = {Towards a Unifying Framework for Formal Theories of Novelty},
    booktitle = {AAAI Conference on Artificial Intelligence (AAAI 2021)},
    year = {2021},
    }

New Research in arXiv / bioRxiv

  1. "On the Effectiveness of Image Manipulation Detection in the Age of Social
    Media,"
    Rosaura G. VidalMata, Priscila Saboia, Daniel Moreira, Grant Jensen,
    Jason Schlessman, Walter Scheirer,
    April 2023.
    [pdf]
    [bibtex]
    @misc{vidalmata2023effectiveness,
    title={On the Effectiveness of Image Manipulation Detection in the Age of
    Social Media},
    author={Rosaura G. VidalMata and
    Priscila Saboia and
    Daniel Moreira and
    Grant Jensen and
    Jason Schlessman and
    Walter J. Scheirer},
    year={2023},
    eprint={2304.09414},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
    }
  2. "Has the Virtualization of the Face Changed Facial Perception?
    A Study of the Impact of Augmented Reality on Facial Perception,"
    Louisa Conwill, Samuel Anthony, Walter Scheirer,
    March 2023.
    [pdf]
    [bibtex]
    @misc{conwill2023virtualization,
    title={Has the Virtualization of the Face Changed Facial Perception?
    A Study of the Impact of Augmented Reality on Facial Perception},
    author={Louisa Conwill and Samuel Anthony and Walter Scheirer},
    year={2023},
    eprint={2303.00612},
    archivePrefix={arXiv},
    primaryClass={cs.HC}
    }
  3. "Human Activity Recognition in an Open World,"
    Derek Prijatelj, Samuel Grieggs, Jin Huang, Dawei Du, Ameya Shringi, Christopher Funk,
    Adam Kaufman, Eric Robertson, Walter Scheirer,
    December 2022.
    [pdf]
    [bibtex]
    @misc{https://doi.org/10.48550/arxiv.2212.12141,
    author = {Prijatelj, Derek S. and
    Grieggs, Samuel and
    Huang, Jin andi
    Du, Dawei and
    Shringi, Ameya and
    Funk, Christopher and
    Kaufman, Adam and
    Robertson, Eric and
    Scheirer, Walter J.},
    title = {Human Activity Recognition in an Open World},
    publisher = {arXiv},
    year = {2022}
    }
  4. "Using Human Perception to Regularize Transfer Learning,"
    Justin Dulay, Walter Scheirer,
    November 2022.
    [pdf]
    [bibtex]
    @misc{Dulay2022b,
    doi = {10.48550/ARXIV.2211.07885},
    url = {https://arxiv.org/abs/2211.07885},
    author = {Dulay, Justin and Scheirer, Walter J.},
    title = {Using Human Perception to Regularize Transfer Learning},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution Share Alike 4.0 International}
    }
  5. "Guiding Machine Perception with Psychophysics,"
    Justin Dulay, Sonia Poltoratski, Till Hartmann, Samuel Anthony,
    Walter Scheirer,
    July 2022.
    [pdf]
    [bibtex]
    @misc{Dulay2022,
    url = {https://arxiv.org/abs/2207.02241},
    author = {Dulay, Justin and
    Poltoratski, Sonia and
    Hartmann, Till S. and
    Anthony, Samuel E.
    and Scheirer, Walter J.},
    title = {Guiding Machine Perception with Psychophysics},
    publisher = {arXiv},
    year = {2022}
    }
  6. "On the Objective Evaluation of Post Hoc Explainers,"
    Zachariah Carmichael, Walter J. Scheirer,
    June 2021.
    [pdf]
    [bibtex]
    @misc{carmichael2021objective,
    title={On the Objective Evaluation of Post Hoc Explainers},
    author={Zachariah Carmichael and Walter J. Scheirer},
    year={2021},
    eprint={2106.08376},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
    }

Selected Projects (see all)

Practical Technology Ethics

Practical Technology Ethics

Fall 2019 - Present

Constructive criticism of emerging technologies, scientific trends, and policies

Keywords: machine learning, facial analysis, election integrity, scientific integrity, peace studies
Human-Drone Partnerships for Emergency Response

Human-Drone Partnerships for Emergency Response

Fall 2019 - Present

Semi-autonomous UAVs to support emergency response scenarios, such as fire surveillance and search and rescue

Keywords: drones, software engineering, participatory design, object detection, first responders
Unconventional Approaches to Hacking Neural Networks

Unconventional Approaches to Hacking Neural Networks

Spring 2018 - Present

Uncovering new vulnerabilities in neural network-based applications where security is a consideration

Keywords: machine learning, neural networks, face recognition, security, backdoors, exploits
Hardware-Aware Distributed Hyperparameter Optimization

Hardware-Aware Distributed Hyperparameter Optimization

Spring 2017 - Present

Intelligent mapping of hardware resources to model search tasks

Keywords: machine learning, hyperparameters, optimization, distributed computing, connectomics
Computational Models of Zebrafish Sensory Integration

Computational Models of Zebrafish Sensory Integration

Fall 2016 - Fall 2020

Modeling neural circuits to explain fish behavior and apply to generalized recognition problems in AI

Keywords: computational neuroscience, zebrafish, sensory integration, data fusion, extreme value theory
Image Restoration and Enhancement for Visual Recognition

Image Restoration and Enhancement for Visual Recognition

Fall 2016 - Present

Work to bridge the gap between computational photography and visual recognition

Keywords: deblurring, denoising, super-resolution, deep learning, object recognition, video
Face Synthesis

Face Synthesis

Fall 2015 - Present

Fast and accurate photo-quality face synthesis for use in face recognition work

Keywords: face synthesis, face recognition, face analysis, computer graphics, data augmentation
Learning-free Segmentation

Learning-free Image Segmentation

Fall 2015 - Spring 2019

Image segmentation for settings where machine learning-based approaches are slow and do not generalize

Keywords: image filtering, volumetric segmentation, connectomics, medical imaging, iris recognition
Tools for Neuroscience

Tools for Neuroscience

Fall 2015 - Fall 2020

Techniques to help neuroscientists understand neuroanatomy and function

Keywords: connectomics, electron microscopy, two-photon imaging, x-ray tomography, behavior
Brain-informed Machine Learning

Brain-informed Machine Learning

Fall 2014 - Spring 2019

Applying measurements of neural activity to regularize machine learning models

Keywords: fMRI, psychophysics, classification, deep learning, neural-weights, object recognition
Visual Place Recognition

Visual Place Recognition

Spring 2013 - Fall 2019

Condition invariant visual place recognition for robot navigation inspired by vision science

Keywords: robotics, navigation, place recognition, whole image matching, patch matching, calibration
Is good recognition metric?

Good Recognition is Non-metric

Fall 2012 - Spring 2014

A new look at a fundamental question in computer vision: is recognition metric?

Keywords: machine learning, metric learning, recognition, face recognition, object recognition
Psychophysics

Psychophysics for Computer Vision

Fall 2012 - Present

Measuring exemplar-by-exemplar difficulty and the pattern of errors of humans for supervised learning

Keywords: machine learning, psychology, citizen science, psychophysics, face detection, attributes
Open Set Recognition

Open Set Recognition

Spring 2011 - Present

Theory and algorithms that address the difficult problem of training without complete class knowledge

Keywords: machine learning, object recognition, face recognition, support vector machines, 1-vs-set machine
Extreme Value Theory for Visual Recognition

Extreme Value Theory for Visual Recognition

Spring 2008 - Present

The theory and practice of recognition score analysis for prediction and fusion

Keywords: calibration, meta-recognition, score analysis, object recognition, face recognition, attributes
Language and Literature

Language & Literature

Spring 2009 - Present

Machine learning and related statistical methods to improve the process by which intertextuality is studied

Keywords: computational linguistics, intertextuality, sound, stylistics, classics, authorship attribution
Digital Image and Video Forensics

Digital Image and Video Forensics

Spring 2008 - Present

Approaches to steganalysis, forgery detection and sensor fingerprinting

Keywords: forensics, forgery detection, data hiding, source identification, sensor fingerprinting

Selected Talks (see all)

Activities