This dissertation presents two experiments studying the statistical properties of the undulator radiation in the Integrable Optics Test Accelerator (IOTA) storage ring at Fermilab. The first experiment studies the turn-to-turn fluctuations in the power of the radiation generated by an electron bunch (1--3 billion electrons). Generally, these turn-to-turn fluctuations depend on the full 6D phase-space distribution of the electron bunch. This effect is related to the interference of fields radiated by different electrons. Changes in the relative electron positions and velocities inside the bunch result in fluctuations in the total emitted energy per pass in a synchrotron radiation source. This dissertation presents the most complete (to date) theoretical description of this effect. The experiment in IOTA confirms that the fluctuations depend on the shape, size, and angular divergence of the electron bunch. It reveals the possibility to measure some electron bunch parameters via the fluctuations. The bunch length has been measured by this method in previous experiments. In IOTA, it is shown that it is also possible to measure some transverse properties of the electron bunch distribution (size, angular divergence). This non-invasive electron beam diagnostic technique may be particularly beneficial for the existing and next-generation low-emittance high-brightness ultraviolet and x-ray synchrotron light sources. The second experiment studies the photon statistics of the undulator radiation generated by a single electron circulating in the ring. When there is only one electron, any classical interference-related collective effects are eliminated, and the quantum fluctuations can be studied in detail. In this experiment, on average, there is only one photocount per several hundred revolutions. The collected data are analyzed to find possible deviations from the expected Poisson process exhibiting uncorrelated detection events. In addition, the arrival times of the photocounts are used to track the longitudinal motion of the single electron and to compare it with the simulation. This allows to determine several useful parameters of the storage ring.