This dissertation is about inference problems for general nonlinear functionals of time-varying covariance matrices using high-frequency observations, under a class of nonparametric multivariate It\^o semimartingale models. Two distinctive approaches are investigated, which employ time-domain and frequency-domain techniques. The time-domain technique adopts the pre-averaging method to attenuate noise and utilizes truncation to deal with possible sample-path discontinuities. The frequency-domain technique computes the discrete Fourier transform of sample-path increments which allows for asynchronous observations and missing data, and then estimates the Fourier coefficients of covariance matrix by finite-order Bohr convolution. Given instantaneous covariance matrix estimates, the functional estimation is carried out by evaluating the functional of interest at these estimates. In case the functional is nonlinear, second-order bias correction becomes necessary in order to achieve the optimal convergence rates in various settings. Closed-form expressions of bias, asymptotic variance and their estimators are available. Applications are considered. Particularly, the attention is focus on factor models by principal component analysis. The author demonstrates how to combine the general results in this dissertation with matrix calculus to conduct statistical inference of realized principal component analysis for non-stationary noisy high-frequency data. Main results are (1) nonparametric plug-in frameworks for functional estimation; (2) theoretical guidance on the choice of tuning parameters; (3) bias corrections in the light of higher-order derivatives; (4) statistical theories on consistency, convergence rates, asymptotic normality, efficiency; (5) statistical uncertainty quantification for non-stationary factor analysis; (6) an empirical analysis of large high-frequency panel data spanning 16 years. Besides principal component analysis, this dissertation provides a methodological foundation for inference of continuous-time regression models, Laplace transform, generalized method of moments and specification tests, as well as statistical uncertainty quantification. The work presented here extends theories and methodologies of previous literature to more empirically realistic settings by solving non-trivial statistical challenges posed by noisy data and asynchronous observations.