FAST NUMERICAL LINEAR ALGEBRA FOR GENERATIVE MODELING

Generative modeling aims to learn the underlying probability distribution of observed data and to generate new samples that resemble the training data. In recent years, deep neural networks have achieved remarkable success in high-dimensional data generation tasks, particularly in image and audio generation. Despite the success, many modern architectures rely on overparametrized neural networks, which can be computationally expensive and difficult to interpret and reveal the intrinsic structure of the underlying distribution. In this thesis, we investigate an alternative framework for generative modeling based on techniques from numerical linear algebra. Chapters~2--4 focus on density estimation and develop a series of methods based on tensor network representations. Tensor networks provide structured and interpretable representations for high-dimensional functions, offering significant advantages in terms of computational efficiency and scalability. In Chapter~2, we introduce an efficient variance-reduction scheme that improves upon the standard higher-order singular value decomposition, enabling its application to moderately high-dimensional problems. Chapter~3 presents a tensor-train-based density estimation framework, where a novel convolution--deconvolution strategy is proposed to mitigate the exponential variance growth associated with empirical density estimation in high dimensions. In Chapter~4, we extend these ideas to hierarchical tensor-train representations. The ultimate goal of the series is to achieve linear computational complexity in the dimensionality and maintain high accuracy in many applications. Chapters~5 and~6 focus on diffusion-based approaches. In Chapter~5, we develop a numerical linear algebra framework for diffusion models that enables optimization-free score estimation. Chapter~6 further extends this approach by incorporating sparse regression techniques to recover sparse structures. Extensive numerical experiments demonstrate the effectiveness of the proposed methods, and theoretical analysis establishes error bounds that characterize their dependence on the dimensionality.