Geometry has become a primary language for physicists to describe condensed matter, stem- ming from our natural inclination to think visually. The development of colloidal model sys- tems has created new possibilities for visualizing complex geometries and dynamics in a vast range of material phases. In this thesis, we will use colloidal model systems and geometry as tools for making sense of the underlying physics of material phases. We begin by investigating the use of geometry for guiding the assembly of shaped colloids interacting with depletion forces. Here the shape of the colloids, called superballs, interpolate between two familiar forms: spheres and cubes. We find an interplay between depletant size and colloid shape determines the assembly. In one regime of depletant size, we observe the transition from hexagonal to rhombic crystals consistent with the densest packings. As we tune the size, however, the particles assemble into square lattice structures instead of such dense packings. We explain the assembly with simple geometric considerations. By using a mixture of depletants, one of which is size tunable, we then induce solid-to-solid phase transitions between these phases. Our results introduce a general scenario where particle building blocks are designed to assemble not only into their maximum density states, but also into depletion-tunable interaction-dependent structures. We then move to investigate the influence of crystal assembly from the geometry not of the particles themselves, but of the underlying environment. In particular, we immerse inter- acting particles which would otherwise form a perfect crystal into spatially varying potentials, where they arrange into fascinating distorted structures. Using colloidal experiments and molecular dynamics simulations, we show the external fields have the effect of placing the lattice on a surface with Gaussian curvature, in which it is structurally frustrated. Using this curvature, we are able to predict the distribution of topological defects that emerge to resolve this frustration. Our results show that when repulsive particles are confined by external fields, topological defects are an almost inevitable feature of the ground state. Finally, we establish a new material by driving the individual component building blocks. We achieve this by spinning a system of colloidal magnets in an external magnetic field, forming a cohesive material that behaves like a liquid. Here, we utilize the geometry of the fluid at large, namely the shape of its surface. By understanding how this surface evolves in time, we are able to unlock the physics of the underlying flows. We find that the driving of the particles results in the breaking of time-reversal and parity symmetries, giving way to the emergence of two key features. The first is an odd stress which drives driving uni-directional surface waves and instabilities, with no counterpart in conventional fluids. The second is an anomalous transport coefficient known as odd (or Hall) viscosity, an experimentally long sought property of chiral fluids.