Files
Abstract
Research in model robustness has a long history. Improving model robustness generally refers to the goal of ensuring machine learning models are resistant to a variety of imperfect training and testing conditions. With the unprecedented progress in deep learning architectures, large-scale training, and learning algorithms, pretrained models have become pivotal in AI. However, when considering real-world scenarios, these models are still fragile and brittle, which impedes the safe deployment of NLP models in production systems.
In this work, we consider wide applications in NLP and define model robustness to broader aspects: (1) data-efficient: models can adapt to new domains with limited annotated data in both pretrained-finetuned and trained-from-scratch set-ups; (2) resilient: models can perform reliably under uncertainties and challenging circumstances; (3) fair: predictors or generators can make safe decisions and filter undesirable biases especially those imbued with toxicity, hate, and social bias; (4) trusted: models are expected to yield factual and faithful content.
To tackle these robustness issues, on the modeling side, we explored both discrete and continuous latent-variable generative models and various graphical model configurations; on the learning algorithms side, we investigated generative pretraining and various discriminative finetuning objectives in generative classifiers, gradient-based optimization, and the importance-sampled log marginal likelihood on learning deep latent-variable models; on the applications side, we developed document classifiers, textual relation predictors, a controllable story generator, and a hallucinated content detector.