Traditionally so-called classifiers in productively formed verbs of motion and location in sign languages are said to reflect the semantic class of the moving or located entity. The verbs can include one of several different classifiers about humans and anthropomorphic beings. In this article, we examine phonological and semantic factors that influence the choice of classifier in elicited narratives in two sign languages, Danish Sign Language and American Sign Language. An analysis of classifiers expressed by handshapes of different phonological complexity shows quantitative differences between the two languages, but no influence on the choice of classifier depending on their phonological features. By contrast, a comparison of classifiers used about types of events in the languages reveals that the seemingly identical classifiers have language-specific meanings despite their iconicity. Nevertheless, they share features: hand-internal movement turns out to be statistically significantly correlated with voluntary actions in both languages. Two analyses of the verbs’ sequential behavior, in serial verb constructions and in verb doubling, show no differences between the classifiers used in either language as well as many structural similarities between the languages. The clearest difference between the languages is found at the lexical (unpredictable) level.