In Chapter 2, we consider a limited-memory multiple shooting method for weakly constrained variational data assimilation. Maximum-likelihood-based state estimation for dynamical systems with model error raises computational challenges in memory usage due to the much larger number of free variables. To address this challenge, we present a limited-memory method for estimation of state space models. We prove that the method is stable for increasing time horizon. We demonstrate our findings with simulations in different regimes for Burgers' equation. In Chapter 3, we investigate a temporal decomposition approach to long-horizon dynamic optimization problems. The problems are discrete-time, linear, time-dependent and with box constraints on the control variables. We prove that an overlapping domains temporal decomposition, while inexact, approaches the solution of the long-horizon dynamic optimization problem exponentially fast in the size of the overlap. The resulting subproblems share no solution information and thus can be computed independently in parallel. Our findings are demonstrated with a small, synthetic production cost model with real demand data. In Chapter 4, we investigate the behavior of maximum likelihood estimators (MLEs) of parameters of a Gaussian process with squared exponential covariance function when the computer model has some simple deterministic form. We prove that for regularly spaced observations on the line, the MLE of the scale parameter converges to zero if the computer model is a constant function and diverges to infinity for linear functions. For some commonly used test functions, we compare MLE with cross validation in a prediction problem and explore the joint estimation of range and scale parameters. In Chapter 5, we consider modeling and predicting observations generated from a nonlinear circuit. We consider time series data which is measured from an electrical circuit that exhibits chaotic behaviors. We investigate the performances of Gaussian process and neural network models in short-term prediction and capturing the long-term dynamics. We also explore the effects of different types of model and observational errors on the likelihood function of the initial state with simulations.