Current and next-generation large-scale structure surveys of the Universe are providing detailed measurements at extremely low levels of statistical uncertainty, accompanied by high resolution data. Commensurate numerical simulations that capture the extensive dynamic range of cosmological scales are required to provide accompanying theoretical predictions. Simulating baryonic effects on structure formation, in particular, is increasingly important to sufficiently model cosmological probes within the constricting precision demanded. Historically, smoothed particle hydrodynamics (SPH) simulations have been a popular choice to resolve gas physics, owing to the numerical benefits of a Lagrangian particle-based methodology. We present a formulation of SPH that utilizes a first-order consistent reproducing kernel, a smoothing function that exactly interpolates linear fields with particle tracers. Previous SPH implementations using reproducing kernel (RK) interpolation have had difficulties maintaining conservation of momentum due to the fact that the RK kernels are not, in general, spatially symmetric. Here, we utilize a reformulation of the fluid equations such that mass, linear momentum, and energy are all precisely conserved without any assumption about kernel symmetries, while additionally maintaining approximate angular momentum conservation. Our approach starts from a rigorously consistent interpolation theory, where we derive the evolution equations to enforce the appropriate conservation properties, at the sacrifice of full consistency in the momentum equation. Additionally, by exploiting the increased accuracy of the RK method's gradient, we formulate a simple limiter for the artificial viscosity that reduces the excess diffusion normally incurred by the ordinary SPH artificial viscosity. Collectively, we refer to our suite of modifications as Conservative Reproducing Kernel SPH, or CRKSPH. CRKSPH retains many benefits of traditional SPH methods (such as preserving Galilean invariance and manifest conservation of mass, momentum, and energy) while improving on many of the shortcomings, particularly the overly aggressive artificial viscosity and zeroth-order inaccuracy. We compare CRKSPH to two different modern algorithms (pressure-based SPH and compatibly differenced SPH), demonstrating the advantages of our new formulation when modeling fluid mixing, strong shock, and adiabatic phenomena. We further analyze non-radiative cosmology simulations, contrasting our results to both traditional and modern hydrodynamic schemes -- establishing the efficacy of CRKSPH within the cosmological domain.