The on-demand usage of enterprise software services from the cloud rapidly evolves towards a viable IT outsourcing option. Although the successful use of software services considerably depends on the ability of the consumers to assess the various offerings and select the ones best suited, literature provides little support for the evaluation of software services and their providers. In this manuscript, we address the question of how to support the evaluation of software service providers. Building upon a design science research approach and a literature survey, we propose an assessment framework that assembles relevant criteria for the evaluation of software service providers. We examine the practical relevance of the assembled criteria using the results of an empirical study, in which we surveyed 28 experts on the subject matter. The results indicate that the framework is effective in supporting the assessment of service providers.