Communications of the Association for Information Systems


Evaluations of participant samples for experiments in information systems research often appear to be informal and intuitive. Appropriate participant choice becomes a more salient issue as the population of information technology professionals and users grows increasingly diverse, and the distribution of relevant characteristics in participant samples such as age, gender, nationality, and experience can often be unrepresentative of the characteristics’ distribution in target populations. In this paper, we present a framework based on widely accepted standards for evaluating participant choice and providing rationale that the choice is appropriate. Using a step-by-step approach, we compare current practice in experimental studies from top information systems journals to this framework. Based on this comparison, we recommend how to improve the treatment of participant choice when evaluating the validity of study inferences and how to discuss the tradeoffs involved in choosing participant samples.