Statistical recovery in high-dimensional statistics and signal processing often requests a determination of multiple structured signals from massive data. And depending on its application, either one or both of the signals may be of primarily interest. While most classical statistical techniques focus on the recovery of a single signal with other parameters being pre-fixed, the recent advances in mathematical and computational tools have facilitated the development of estimating multiple structured signals simultaneously. This thesis details several such problems with different goals of signal recovery. ,Chapter 2 describes a low-rank + sparse decomposition problem under data compression and we study rigorous statistical performance guarantee that is achievable using a joint convex optimization based estimator. It is well known that the convex relaxation of the structural constraints leads to a large bias on the strong signals, although it affords computationally tractable algorithms. Chapter 3 develops a new notion of local concavity coefficients to directly handle nonconvexity of structural constraints. Based upon these coefficients, Chapter 4 analyzes convergence of alternating minimization when nonconvex constraints are placed on each of the variables. The theory developed here is general enough to encompass a broad class of multiple structured statistical models such as low-rank + sparse decomposition, multitask regression, and Gaussian factor model under a single framework. Chapter 5 discusses a simultaneous framework for the calibration of an imaging system and image reconstruction in CT imaging. As a preliminary work, an efficient optimization-based approach is proposed for spectrum estimation.