Recovering Structure from Observational Data: Representation, Simulation, and Stability

Modern statistical learning increasingly begins from data that were observed rather than designed. Such data are records of an uncontrolled process. They may contain rich information about the environment that produced them, but they do not by themselves say what should be learned, which variation is useful, or which parts of the data-generating process can be trusted for inference. The central problem is therefore interpretive as well as computational.

We develop this view through three studies of observed environments. The first analyzes diffusion-based generative modeling as a way to represent an observed data distribution. Empirically, these models have been shown to capture features of an environment that are difficult to describe parametrically: dependence, heterogeneity, multimodality, and geometric structure. We ask what makes this possible statistically, and how the denoising operations used by diffusion models recover structure in the underlying distribution. The second study asks what becomes possible once observed environments can be accessed by simulation. Using learned conditional generators, we develop a simulation-based approach to conditional moment restrictions, with nonlinear instrumental variables as the leading example. The third study considers what happens when the observed environment changes. It examines prediction under concept shift, where unobserved environment-specific variation can make a predictor learned in one environment fail to transfer to another. Together, these studies examine how modern learning procedures can be used to recover, simulate, and stabilize structure in observational data.