Plastic deformation is an irreversible change in form, occurring when a material internally reconfigures to relieve mechanical stress. We study plastic flow in granular materials, where the change in state proceeds via particle rearrangements that occur over a broad range of length and time scales. When subjecting a granular material to a constant rate of deformation, it restructures via rapid cascades of particle movement thought to be universal with quake events in other amorphous materials. We 3D-print granular packings composed of different particle shapes, and measure the distribution of cascade magnitudes during uniaxial compression. We find that the characteristic size of the rearrangement cascades can be varied by over two orders of magnitude with particle shape. The underlying physics of the cascades, as measured by the power law exponent of their magnitude distribution, remains invariant. Its value is measured to be more compatible with a mean-field theory of plastic deformation than with one which accounts for the Eshelby treatment of stress redistribution in an elastic medium. When deformation stops and a granular system is held at a constant strain, it is found to relax approximately logarithmically in time, suggesting a complex inner state capable of storing memory. Via alternating compression and decompression steps we are able to imprint multiple memories that play themselves out through nonmonotonic stress relaxation. We are able to explain many aspects of the behavior with a model recently applied to the same memory behavior in crumpled elastic sheets. The model, which approximates the system as an ensemble of simple relaxing elements with broadly distributed timescales, becomes inaccurate when fitting to the form of the relaxation. The nature of the inaccuracy has implications for a wide class of systems exhibiting anomalous, non-Debye relaxation. Finally, we observe granular rearrangement events during the stress relaxation, suggesting the complex inner state is that of glacially slow plastic flow continuing hours past the cessation of compression. The various parts to this story show that a granular material adapts in ways large and small, fast and slow, and that the process has fundamental and far-reaching commonalities with many other systems in nature.