Interspecific aggression amongst nonhuman primates is rarely observed and has been mostly related to scenarios of resource competition. Interspecific infanticide is even rarer, and both the ultimate and proximate socio-ecological factors explaining this behavior are still unclear. We report two cases of interspecific infanticide and five cases of interspecific infant-directed aggression occurring in a well-habituated primate community living in a fragmented landscape in Colombia. All cases were initiated by male brown spider monkeys (Ateles hybridus) and were directed toward infants of either red howler monkeys (Alouatta seniculus: n = 6 cases) or white-fronted capuchins (Cebus albifrons: n = 1 case). One individual, a subadult spider monkey male, was involved in all but one case of interspecific infanticide or aggression. Other adult spider monkeys participated in interspecific aggression that did not escalate into potentially lethal encounters. We suggest that competition for food resources and space in a primate community living in high population densities and restricted to a forest fragment of ca. 65 ha might partly be driving the observed patterns of interspecific aggression. On the other hand, the fact that all but one case of interspecific infanticide and aggression involved the only subadult male spider monkey suggests this behavior might either be pathological or constitute a particular case of redirected aggression. Even if the underlying principles behind interspecific aggression and infanticide are poorly understood, they represent an important factor influencing the demographic trends of the primate community at this study site.