Large Language Models for Open-Ended Domains

Recent advancements in large language models have concentrated on tasks with automatic verifiers, such as code and mathematics. In many domains there is no such verifier, and what counts as a good output rests on tacit knowledge or judgment that resists measurement. This dissertation develops abstractions that make such judgment measurable, then uses them to build and evaluate language models across open-ended domains.

To measure economic narratives, this dissertation defines causal micro-narratives as sentence-level explanations of a cause or effect and builds an inflation-specific ontology, then fine-tunes a model to detect them at scale. Across a century of U.S. newspaper coverage (4.2 million sentences), social and political impact narratives predict disagreement in household inflation expectations 1.8 times more strongly than realized inflation, and lower-income households are 4.6 times more sensitive to them.

To evaluate summaries of complex legal documents, it introduces \dataset, a long-context summarization dataset of U.S. Supreme Court opinions paired with their official syllabuses, and compares automatic metrics, LLM judges, and expert rankings. LLM-based judges do not track expert judgment better than automatic metrics, and fluent summaries that experts prefer can contain factual errors that automatic methods miss, exposing the limits of LLM-as-judge for long-context summarization.

To support voters deliberating over ballot measures, it presents CivicChats, a platform and chatbot interface with three discourse modes (neutral Q&A, argumentative, and reflective) grounded in deliberative democracy. In a pre-registered controlled experiment on two live 2026 ballot measures, chatbots designed for deliberative discourse shifted vote distributions and produced written rationales that engaged substantially more with counterarguments than static information.