Many tasks, like process fault detection, disease diagnostic or maintaining security in public places are connected to the process of counting objects from observing image data like camera footage or micro-scopic images. However, the pivotal process of counting objects based on image data is subject to a lot of error sources when done manually. Therefore, a lot of approaches exist that aim to aid the automa-tion of the counting process. Since most of those methods are of black box nature and have to be ap-plied to real world scenarios, we provide a taxonomy that reflects both: the method characteristic as well as the counting problem characteristics in order to derive a systematization of the scientific field and to provide a tool that acts as a set of guidelines for choosing appropriate counting methods for different situations. We first conduct a literature review, where counting problem characteristics and solutions are extracted and later transformed into a taxonomy. We finally showcase the taxonomy using four case studies that are based on publicly available datasets for the sake of scientific comprehension.