Advances in Theories, Methods and Philosophy

Loading...

Media is loading
 

Paper Number

2407

Paper Type

Completed

Description

Behavioral experiments are a highly suitable method for testing theories, as they can establish causality while controlling for other confounding factors. However, researchers that aim to conduct and publish such studies face various concerns about the methodological approach. A lack of clarity exists in our field as to which related practices and design decisions are legitimate and accepted. To address this issue, we present a structured literature review that analyzes the designs of 168 behavioral experiments published in the Senior Scholars’ Basket of journals. We find that most experiments are confirmatory, individual-level, between-subjects laboratory experiments. At the same time, we find that some under-represented experiment designs, such as exploratory online experiments, may bear potential for identifying new behaviors and constructing new or proper-to-IS theories. This paper contains an in-depth discussion on the findings and provides decision support to IS researchers that seek to design and publish behavioral experiments.

Comments

20-Methods

Share

COinS
Best Paper Nominee badge
 
Dec 12th, 12:00 AM

Design Decisions in Behavioral Experiments: A Review of Information Systems Research

Behavioral experiments are a highly suitable method for testing theories, as they can establish causality while controlling for other confounding factors. However, researchers that aim to conduct and publish such studies face various concerns about the methodological approach. A lack of clarity exists in our field as to which related practices and design decisions are legitimate and accepted. To address this issue, we present a structured literature review that analyzes the designs of 168 behavioral experiments published in the Senior Scholars’ Basket of journals. We find that most experiments are confirmatory, individual-level, between-subjects laboratory experiments. At the same time, we find that some under-represented experiment designs, such as exploratory online experiments, may bear potential for identifying new behaviors and constructing new or proper-to-IS theories. This paper contains an in-depth discussion on the findings and provides decision support to IS researchers that seek to design and publish behavioral experiments.

When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.