Despite being a topic of discussion among researchers for many years, measurement of information system project success (ISPS) is still an unsolved challenge. Scholars in information systems as well as general project management literature propose various approaches for assessing this concept. Instead of seeking a generally accepted measurement approach, some scholars suggest to assess ISPS by applying situational models, that is using different success dimensions depending on situational variables like stakeholder perspective or project type. This article outlines on-going research concerning relevant dimensions of ISPS depending on specific project characteristics. We conduct a quantitative empirical study using questionnaires among information system project managers. We continue existing research on ISPS measurement by providing statements about the relevance of potential success dimensions for project characteristics like contract type or type of contracting organisation. Preliminary results collected from 53 projects indicate that certain dimensions (e.g. meeting functional requirements and customer satisfaction) are consistently considered relevant regardless of project characteristics, whereas the importance of other dimensions (e.g. adherence to schedule) varies depending on such characteristics. We aim at contributing to a more adequate assessment of ISPS using situational models, which is of high relevance for both researchers and practitioners.