Most processes in nature take place out of equilibrium. Storage and processing of information is an important process in many contexts starting from the brains of living organisms to the storage of genetic information in our cells. These biochemical processes consume energy and function far away from equilibrium. During my PhD, I explored how one could improve the information processing capability of such systems by taking them out of equilibrium. I primarily explored it in the context of associative memory and generative diffusion models.