Despite its generic title, this thesis is about a specific notion of sparsity, the one introduced by McCullagh and Polson (2018). In that paper, the intuitive idea that sparsity, in a statistical framework, refers to those ''phenomena that are mostly negligible or seldom appreciably large'', has, for the first time, been given a mathematical definition. In studying this definition of statistical sparsity as a limiting property of a sequence of probability distributions, research has proceeded along different lines, which nevertheless intersect at all times. In all cases, our work has been driven by both theoretical and practical motivations. The notion of negligibility, for instance, is developed from the necessity of describing the behavior of a sparse distribution in a region around zero, a necessity which is commonly encountered in applied work. At the same time, doing this in a mathematical way, allows us to define very clearly what is the perimeter within which this notion is informative, and can be used. Another main direction of research we pursue, aims at extending the definition of sparsity to distributions which are defined on d-dimensional vector spaces, d>1. Within this framework, we consider two scenarios: in the first one, the d-dimensional measure is a product of d one-dimensional sparse measures; in the second one, instead, the d-dimensional measure is rotationally invariant with respect to the inner product imposed on R^d, and sparsity is driven by the radial component. For both cases, we develop some theory as well as present how this theory can be in fact applied in the context of various statistical problems.