There is a well-developed asymptotic theory for sample means and sample second-order statistics of low dimensional stationary processes. However, many important problems on their asymptotic behaviors are still unanswered for time series which can be high-dimensional, nonstationary and non-Gaussian.,This thesis concerns the estimation and inference of high-dimensional time series under the framework of functional dependence measure. We first consider the problem of approximating sums of high dimensional stationary time series by Gaussian vectors. We also consider an estimator for long-run covariance matrices and study its convergence properties. Our results allow constructing simultaneous confidence intervals for mean vectors of high-dimensional time series with asymptotically correct coverage probabilities. As an application, we can do simultaneous inferences for covariance matrices of high-dimensional stationary time series. We also propose a Kolmogorov-Smirnov type statistic for testing distributions of high-dimensional time series.,This thesis also presents a systematic asymptotic theory for the estimates of time-varying second-order statistics for a general class of high-dimensional nonstationary processes. In particular, we investigate the estimation of time-varying autocovariance matrix functions, spectral density matrices and coherence matrices for high-dimensional locally stationary processes. Besides, we use,the constrained $\ell_1$ minimization approach to estimate the inverse of the,spectral density matrix which can be used to identify the graphical structure for high-dimensional locally stationary processes. We derive the convergence rates of the estimates which depend on the sample size, the dimension, the moment condition and the dependence of the underlying processes.