In the modern era of machine learning, where increasingly complex models drive decision-making, statistical inference becomes challenging. Assumption-lean inference minimizes reliance on distributional assumptions, ensuring robustness across diverse settings. This thesis explores two key aspects—predictive inference via conformal prediction and conditional independence testing—each providing practical, interpretable solutions to real-world challenges. The first part investigates distribution-free predictive inference. Conformal prediction (CP) constructs prediction intervals with distribution-free guarantees, but its validity relies on exchangeability and only ensures marginal coverage, leading to failures in non-i.i.d. settings, undercoverage for certain subpopulations, or discrepancies when test and training distributions differ. We address these limitations in two steps. First, in survival analysis, where right-censoring restricts observation of true survival times, we propose a novel lower prediction bound using a data-adaptive filter that excludes low censoring times, ensuring valid marginal coverage under mild conditions. Next, we tackle local coverage, aiming for prediction intervals that retain validity for each feature configuration. While exact local coverage is theoretically impossible, we introduce randomly-localized conformal prediction (RLCP), a framework that provides interpretable guarantees for relaxed notions of local validity, such as approximate coverage under smooth covariate shifts or within sufficiently large subpopulations. The second part develops assumption-lean tests for conditional independence. Given the inherent difficulties of testing conditional independence in a distribution-free manner, we propose a test for the conditional independence of X and Y given Z, under the assumption that X is stochastically increasing in Z. Our procedure, PairSwap-ICI, offers significant flexibility while ensuring finite-sample Type I error control and achieving high power against a broad range of alternatives that deviate sufficiently from the null.