The human has never been and will never be separated from their physical environment. How does the physical environment shape us, our feelings and thoughts, and our behavior? In this dissertation, we present a variety of research motivated by these questions. We focus on two semantic dimensions of the physical environment that are pertinent to this topic—its level of order/disorder and its level of builtness/naturalness. In Chapter 1, we examine whether low-level visual features that influence perceived disorder (visual disorder cues) have been overlooked in research on broken windows theory that assumes that social disorder cues and complex social reasoning are necessary for the effect of disorderly environments on rule-breaking behavior. In one set of experiments, we identified key visual disorder cues that generalize across a variety of visual stimuli with a variety of semantic content. In another set of experiments, we demonstrated that visual disorder can encourage cheating. In an additional two experiments, we explored mechanisms of this effect. In Chapter 2, we identify and resolve a paradoxical relationship between disorder and naturalness (if disorder is aesthetically aversive and naturalness is aesthetically pleasing, how is it that natural environments are disorderly?). Across four sets of experiments, we tested three competing hypotheses that could explain this relationship and found that (a) the effects of naturalness and disorder on aesthetic preference are independent and (b) the effect of naturalness on aesthetic preference trumps the effect of disorder. Further, we show that scene semantics are both necessary and sufficient for this nature-trumps-disorder effect, and their interaction with low-level visual features amplifies this effect. In Chapter 3, we examine whether high-level semantics of a scene related to disorder and naturalness can be preserved in the low-level visual features of that scene, contrary to traditional visual perception models that assume that integration of low-level visual features must occur before high-level semantics are perceived. The results of two experiments suggest not only that disorder and naturalness semantics can be preserved in low-level visual features, but also that disorder and naturalness semantics can be preserved in different types of low-level visual features. This research adds to a growing body of literature that suggests that low-level visual features can carry semantic information. Together, these chapters present a variety of evidence substantiating the intimate connection we have to our ever-present physical environments, and provide insight into the linkage between low-level visual processing, semantics, aesthetic sense, and behavior.