Sample-Efficient Adaptive Optimization: From Active Learning to Real-Time Triggers and Prompt Systems

Many problems in modern machine learning require adaptively making a sequence of decisions under tight budgets on data, queries, compute, or latency, each based on the partial information obtained so far. As an example, consider pool-based active learning, where one can request labels for only a small fraction of available data, and the goal is to adaptively select a small batch of unlabeled instances whose annotation will most improve downstream generalization. Another example is automatic prompt engineering for large language models, where each candidate prompt incurs a forward pass on a frontier Large Language Model (LLM), and the goal is to refine the prompt to maximize a downstream utility under both context-length and inference-cost budgets. Understanding how to make such decisions efficiently is fundamental for deploying machine-learning systems at scale, in regimes where each evaluation is too expensive to query freely and the underlying conditions drift faster than offline retuning can keep up. Unfortunately, most of the existing techniques for solving such problems are either domain-specific heuristics which do not strive for optimality and do not generalize across the non-stationarity of the underlying signal, or principled approaches that are difficult to scale to the data volumes, context lengths, or latency budgets at which contemporary systems operate.

To ease the tension between theory and practice, this dissertation pursues the fundamentals of sample-efficient adaptive optimization, with the goal of devising methods that are simultaneously principled (with appropriate guarantees on decision quality or stability), framework-agnostic (deployable into the optimization stacks that practitioners already use), and adaptive (responsive to non-stationarity in the underlying signal). From the methodological perspective, we develop a recurring recipe with three ingredients: (1) pairwise-ranking surrogates that supervise a learned utility model with a noise-robust binary signal rather than a noisy absolute score; (2) stochastic exploration over the upper tail of candidate decisions, structured by the Gumbel-Top-k trick rather than by a hand-tuned temperature; and (3) bilevel or two-stage decompositions that separate fast adaptation to the current iteration’s data from slow generalization across iterations. From the applied perspective, we deploy these methodological tools in three high-stakes production settings: real-time triggering at the Large Hadron Collider, the joint discovery of functional material–device systems, and prompt engineering for text-to-image generation in ads industry.

More specifically, in Part II of this dissertation, we develop our methodological contributions. We first consider pool-based active learning under one-shot acquisition, where prior learning-based methods require thousands or millions of training samples to fit a usable acquisition function, precisely the budget regime that active learning was designed to economize. We recast the surrogate utility model as a learning-to-rank problem over pairs of training subsets, and regularize it via a multi-task bilevel objective with an optimal-transport distance to the validation set as supervision. The resulting framework, RAMBO, outperforms heuristic and learning-based baselines on four image-classification benchmarks under one-shot acquisition, and the gains persist under label noise and class imbalance.

We then turn to textual gradient descent (TGD), a recent family of methods that refine language-model prompts iteratively by treating LLM feedback as a gradient over the prompt. We identify and empirically characterize an implicit context wall that limits the scaling of TGD: performance on a representative reasoning benchmark peaks at fewer than a hundred training examples and degrades thereafter, well below the explicit context-window limit. We propose TSGD-M, a sampling-based momentum extension that replaces concatenation of historical prompts with Gumbel-Top-k sampling over a cache of past prompts weighted by their running validation accuracy, and we provide a Bayes-optimality justification for the resulting screening-then-refinement decision rule. TSGD-M integrates seamlessly into existing prompt optimization frameworks (TextGrad, DSPy-COPRO, AdalFlow) and achieves consistent gains across six benchmarks.

Part III of this dissertation aims to demonstrate the breadth of the methodological program through three application studies. For high-throughput physics, we cast online trigger threshold tuning as a sequential decision problem and solve it with reinforcement learning. A controller ingests streaming summaries of recent rates and signal-sensitive features and updates trigger thresholds to maximize signal efficiency while holding a target background rate within a tolerance band. We adapt Group-Filtered Policy Optimization (GFPO) to streaming control and introduce two feasibility-enforcing variants, GFPO-F and GFPO-FR, that respect the background-rate constraint during training. On a benchmark that emulates realistic collider operation, the learned controller increases the fraction of in-tolerance time intervals by 48% on a transverse-energy (HT ) trigger and 28% on an anomaly-detection trigger, with a cumulative gain of up to 2% in signal efficiency on the in-tolerance intervals. Transferred without fine-tuning from simulation to real CMS collision data (Run 283408), the same controller improves the in-tolerance fraction by 56% and 28% respectively, with further signal-efficiency gains on both triggers. To our knowledge this is the first demonstration of RL-based trigger control on real Large Hadron Collider data. For functional materials– device co-design, we develop Text-Twin-Translation, a full-stack ML framework that couples natural-language representations of materials with simulation-based digital twins of devices to accelerate the joint discovery of material–device systems. For production-level text-toimage generation in ads industry, we develop PRISM-DUEL, a black-box automatic prompt engineering framework that replaces unreliable absolute scoring by zero-shot LLM judges with a dueling-bandit posterior over pairwise preferences from a vision-language judge, and we demonstrate consistent gains in visual faithfulness and prompt interpretability across three text-to-image model families.