This work collects three projects. The broad theme common to all three projects is quantifying uncertainty, preferably under a weak set of assumptions. This theme is explored through mainly two types of problems of statistical inference that exemplify aspects of modern statistics. The first type pertains to the problems of learning about the difference between two graphical models given two sets of independent and identically distributed (IID) observations when the number of variables far exceeds either sample size. In particular, we develop methods for characterizing the differential structure with theoretical guarantees. The second has to do with the problems of predictive inference in an assumption-lean setting. That is to say, we assume that the data are IID and the learning algorithms being used are permutation-symmetric, but we refrain from making additional assumptions. The particular problem we focus on is that of constructing a predictive set for an ensemble prediction with a coverage guarantee that holds non-asymptotically for any data distribution and any choice of the ensemble model. We propose a method that is competitive both in terms of computational cost and statistical efficiency.