Elias Stengel-Eskin

I am a Postdoctoral Research Associate at the University of North Carolina, Chapel Hill in the MURGe-Lab led by Mohit Bansal. I received my Ph.D. in 2023 from Johns Hopkins University, where I was supervised by Benjamin Van Durme and supported by an NSF GRFP.

I aim to develop AI agents that can intelligently communicate and collaborate with people and each other. A central focus of my work is multi-agent communication via text, which has led to work on multi-LLM discussions/debates, pragmatic/verbalized uncertainty, persuasion, and semantic parsing/text-to-code (transforming text into representations of its meaning). I believe that intelligent agents will be multimodal: another line of my work covers multimodal tasks and focuses especially on underspecified tasks (ambiguous examples, long-context inputs). As we scale up tasks, underspecification and ambiguity will become increasingly relevant. I have a long-standing interest in implicit phenomena such as vagueness, underspecification, and ambiguity. While I’ve mostly explored these topics through a linguistic lens, I am interested in their importance to intelligence more broadly.

Concretely, some of the areas I’ve been publishing on recently are:

• Confidence Estimation and Calibration:
  • on pragmatic reasoning for better QA calibration (Stengel-Eskin et al., NeurIPS 2024)
  • on calibration in semantic parsing (Stengel-Eskin and Van Durme, TACL 2023)
  • on balancing safety and usability using confidence (Stengel-Eskin and Van Durme, EMNLP 2023)
• Multi-Agent/Multi-Model Reasoning:
  • on training models to accept good and resist bad persuasion (Stengel-Eskin et al., 2024)
  • on structured distillation for learning from multiple LLM reasoning agents (Chen et al., ICML 2024)
  • on a new benchmark to assess game-theoretic abilities for LLM agents (Duan et al., NeurIPS 2024)
  • on multi-agent iterative coarse-to-fine refinement for reasoning tasks (Chen et al., 2024)
  • on using bandits to select instance-level reward models for LLM alignment (Nguyen et al., 2024)
• Learning Skills and Abstractions for Agents/Coding/Planning:
  • on learning coding abstractions with LLMs (Stengel-Eskin et al., ICML 2024)
  • on discovering skills for RL agents (Fu et al., ICML 2024)
  • on distilling skills from teachers for LLM agents (Hashemzadeh et al., CoLLAs 2024)
  • on hybrid and controllable System 1 and System 2 planning (Saha et al., 2024)
• Ambiguity and Underspecification:
  • on ambiguity in semantic parsing (Stengel-Eskin and Van Durme, ICLR 2024)
  • on improving VQA through preemptive self-clarification (Prasad et al., ICLR 2024)
  • on rephrasing and analyzing ambiguous questions in VQA (Stengel-Eskin et al. ACL 2023)
• Improving Multimodal Models and LLM Agents:
  • on building and testing data generation agents for creating training data (Khan et al., 2024
  • on a tree-based representation for LLM-based video reasoning (Wang et al. 2024)
  • on improving visual prompting/object grounding without training (Wan et al., ECCV 2024)
  • on a more effective/efficient self-consistency method for LLM agents (Wang et al., ACL 2024)
  • on Western cultural bias in VLMs and the effect of pretraining language (Ananthram et al., 2024)
  • on visual commonsense in unimodal and multimodal models (Zhang et al, 2022)

Before starting my Ph.D., I received my B.A.&Sc. with First Class Honours in Cognitive Science from McGill University, focusing in computer science and linguistics. While at McGill, I worked as a research assistant at the Montreal Language Modeling Lab (MLML), now MCQLL supervised by Morgan Sonderegger. I wrote my honours thesis (supervised by Timothy O’Donnell) on a variational inference algorithm for a model of language acquisition.