AIS Transactions on Human-Computer Interaction

ECOVAL: Ecological Validity of Cues and Representative Design in User Experience Evaluations

Suzanne Kieffer — Fri, 23 Jun 2017 13:47:23 PDT

Egon Brunswik coined and defined the concepts of ecological validity and representative design, which are both essential to achieve external validity. However, research in HCI has inconsistently and incorrectly used Brunswik’s concept of ecological validity, which prevents the field from developing cumulative science and from generalizing the findings of user experience (UX) evaluations. In this paper, I present ECOVAL, a framework I built on Brunswik’s ideas. On the one hand, ECOVAL helps HCI researchers describe and assess the ecological validity of cues in UX evaluations. On the other hand, ECOVAL guidelines—formulated as a step-by-step procedure—help HCI researchers achieve representative design and, therefore, increase external validity. An industrial case study demonstrates the relevance of ECOVAL for achieving representative design while conducting formative UX testing. In discussing the case study, I describe how ECOVAL can help HCI researchers assess and increase the validity of UX experiments and generalize UX findings. I also illustrate the trade-offs between internal and external validities and UX resources that inevitably arise when one conducts UX experiments. From the results, I sketch avenues for future research and discuss the related challenges that future work should address.

Decision Confidence and Purchase Intention in Software Trials: A Cognitive Stopping Rule Perspective

Xue Yang et al. — Fri, 23 Jun 2017 13:47:19 PDT

Organizations often offer software trials to potential buyers as a form of promotion. The trial experience provides first- hand information of the product, which contributes to prudent purchase decisions. Despite the popularity of software trials in consumer markets, we have yet to fully understand how individuals formulate purchase decisions throughout the trial process. This study proposes a novel perspective by focusing on 1) decision confidence that individuals establish when using a software trial based on the compatible cognitive stopping rules (CSRs) they apply and 2) the influence of decision confidence on purchase intention. We conducted a controlled lab experiment in which 204 participants tried an interior design software product. We found that the participants formed their decision confidence by applying compatible CSRs during the software trial. Decision confidence augmented the positive influence of product satisfaction on purchase decision. Interestingly, decision confidence attenuated the positive relationship between a user’s satisfactory trial experience and the decision to purchase the software. We provide detailed discussions on our findings, limitations, theoretical contributions, and implications for practice.

Social Media Induced Technostress and its Impact on Internet Addiction: A Distraction-conflict Theory Perspective

Stoney Brooks et al. — Fri, 23 Jun 2017 13:47:14 PDT

Using social media is the most common activity on the Internet, and much research has examined the phenomenon. While the current literature focuses on the positives of using social media, there is a comparative lack of research on its negative effects, especially in the context of the workplace. Research has identified one critical negative impact of contemporary technology as technostress, which refers to stress induced by information and communication technologies. In this paper, we apply distraction-conflict theory (DCT) to the literature on social media, technostress, and addiction to theorize that one can view social media in the workplace as a distraction conflict, which, in turn, can induce technostress and, subsequently, Internet addiction. To test this theoretical model, we conducted a survey on 1731 participants recruited from Mechanical Turk. The survey examined the similarities and differences between two popular social media platforms: Facebook and YouTube. Overall, the results provide support for positive associations between the distraction felt from social media and social media-induced technostress and between social media- induced technostress and Internet addiction. While Facebook and YouTube have similarities, we found notable differences as well. This study contributes to the IS field by using DCT as a novel and valuable lens through which researchers and practitioners can think about the negative effects of using social media at work. The paper also offers insight into implications for research, practice, and future research areas.

How Do Learners Interact with E-learning? Examining Patterns of Learner Control Behaviors

Sandra Fisher et al. — Fri, 23 Jun 2017 13:47:10 PDT

There has been significant debate in the literature on technology-mediated training about the appropriate role of learner control. We define learner control as giving trainees the ability to make choices about how they proceed through the learning environment. We explore two perspectives. First, we consider learners’ stated preferences for the extent of control in the learning environment. Second, we analyze the actual online learning behaviors of 518 trainees in a Fortune 500 organization. We compare a measure of learner control preferences to the most commonly used framework of learner control that comprises five dimensions: pace of instruction, sequence of topics, specific content covered, amount of advice/feedback provided, and type of media. We also compare the dimensionality of learner behaviors to this framework and examine the relationship between learner preferences and learner behaviors. Results suggest that fewer dimensions can capture both learner preferences and behaviors than what the literature currently suggests. Specifically, media control aligned with both pace and content control. The relationship between stated learner control preferences and learner control behaviors was relatively weak. However, we found support for the recently identified dimension of scheduling control and suggest a new learner control dimension of performance control, consistent with the importance of practice retrieval for learning.

Toward a More Secure HRIS: The Role of HCI and Unconscious Behavior

Humayun Zafar et al. — Thu, 30 Mar 2017 01:27:07 PDT

By design, human resource information systems (HRIS) hold confidential and sensitive information. Therefore, one needs to ensure the security of these systems from unintentional mistakes that may compromise such information. Current systems design and training procedures of HRIS unintentionally help reinforce unsecure behaviors that result in non-malicious security breaches. Measures to improve security through design and training may only occur by breaking the use/impact cycle that individuals have habitually formed. Using strong contexts and cues allow trainers to interrupt individuals’ habits. Then, they have the opportunity to enforce the repetition of the desired behavior. This paper introduces a model of habit formation from consumer behavior that one may apply to HRIS.

Conceptual Modeling in Human Resource Management: A Design Research Approach

Stefan Strohmeier et al. — Thu, 30 Mar 2017 01:27:03 PDT

In this paper, we introduce conceptual human resource modeling (CHRM) as a new methodical paradigm in HRM. To this end, we employ a design research approach. On a general level, we identify general tasks that CHRM can tackle and general solutions that it can provide. On a concrete level, we use the specific problem of employee assignment as an example and develop a specific CHRM solution for this HR task. By using the prototypical CHRM tool, we show how one can solve practical HR tasks based on CHRM. Finally, we discuss the lessons learned and implications for future research in CHRM.

The Importance of the Interface between Humans and Computers on the Effectiveness of eHRM

Richard D. Johnson et al. — Thu, 30 Mar 2017 01:26:59 PDT

Technology has had a dramatic impact on the practice of human resources, and its impact is rapidly increasing. Even so, little research has examined how to apply information systems and human-computer interaction principles to designing human resource information systems. In this paper, we focus more closely on the role that the interface between the computer and human play in the success of electronic human resource management. Specifically, we a) briefly review the individual requirements of several eHRM functions (e.g., e-recruiting, e-selection, e-learning, e- compensation/benefits), b) consider how an understanding of human computer interaction can facilitate the success of these systems, c) reviews research on technical issues associated with eHRM, and d) highlight how applying HCI principles can increase their effectiveness. In addition, we introduce the remaining seven papers in the special issue.

The Impact of Sentiment Analysis Output on Decision Outcomes: An Empirical Evaluation

Parisa Lak et al. — Thu, 30 Mar 2017 00:46:59 PDT

User-generated online content serves as a source of product- and service-related information that reduces the uncertainty in consumer decision making, yet the abundance of such content makes it prohibitively costly to use all relevant information. Dealing with this (big data) problem requires a consumer to decide what subset of information to focus on. Peer-generated star ratings are excellent tools for one to decide what subset of information to focus on as they indicate a review’s “tone”. However, star ratings are not available for all user-generated content and not detailed enough in other cases. Sentiment analysis, a text-analytic technique that automatically detects the polarity of text, provides sentiment scores that are comparable to, and potentially more refined than, star ratings. Despite its popularity as an active topic in analytics research, sentiment analysis outcomes have not been evaluated through rigorous user studies. We fill that gap by investigating the impact of sentiment scores on purchase decisions through a controlled experiment using 100 participants. The results suggest that, consistent with the effort-accuracy trade off and effort-minimization concepts, sentiment scores on review documents improve the efficiency (speed) of purchase decisions without significantly affecting decision effectiveness (confidence).

How Social Media Can Enhance Access to Information through Transactive Memory Development

Hossam Ali-Hassan et al. — Tue, 27 Dec 2016 11:47:10 PST

A key challenge for managing talent in organizations is locating and coordinating expertise. In this study, we propose that employees who use social media can help an organization locate knowledge workers who are vital to organizational growth and competitiveness. We draws on transactive memory (TM) theory to examine the relationship between social media use and knowledge workers’ access to information as mediated by the formation of an organization-wide transactive memory. We conducted the research using a mixed-methods approach that combined insights from a qualitative investigation with a confirmatory large-scale survey in a multinational information technology firm. We empirically show that social media use had a positive but indirect relationship with knowledge workers’ access to information via the mediation of the three dimensions of TM. We discuss our findings’ implications for theory and practice, including human resource management, and directions for future research.

Job Applicants’ Information Privacy Protection Responses: Using Social Media for Candidate Screening

John R. Drake et al. — Tue, 27 Dec 2016 11:47:06 PST

For human resource (HR) departments, screening job applicants is an integral role in acquiring talent. Many HR departments have begun to turn to social networks to better understand job candidates’ character. Using social networks as a screening tool might provide insights not readily available from resumes or initial interviews. However, requiring access to an applicants’ social networks and the private activities occurring therein—a practice currently legal in 29 U.S. states (Deschenaux, 2015)—could induce strong moral reactions from the job candidates because of a perceived loss of information privacy. Subsequently, such disclosure requests could induce job candidates to respond in a multitude of ways to protect their privacy. Given that an estimated 2.55 billion individuals will use social media worldwide by 2017 (eMarketer, 2013), the repercussions from requests for access social media environments have potentially far-reaching effects. In this research, we examine how one such disclosure request impacted six information privacy protective responses (IPPRs) (Son & Kim, 2008) based on the job candidates’ perceived moral judgment and the perceived moral intensity of the HR disclosure request. These responses occurred when we asked respondents to provide personal login information during a hypothetical interview. By modeling data derived from a sample of 250 participants in PLS-SEM, we found that the five IPPRs (i.e., refusal, negative word of mouth, complaining to friends, complaining to the company, and complaining to third parties) were all significant responses when one judged the request to be immoral and perceived the moral intensity concept of immediate harm. The amount of variance explained by these five IPPRs ranged from 17.7 percent to 38.7 percent, which indicates a solid initial foundation from which future research can expand on this HR issue. Implications for academia and practice are discussed.

Introduction to the Special Issue on Human Resource Information Systems and Human Computer Interaction

Richard D. Johnson et al. — Tue, 27 Dec 2016 11:47:01 PST

In this special issue, we focus on the role that human-computer interaction (HCI) can play in the development and successful use of human resource information systems (HRIS) in organizations. There is no doubt that information systems have transformed the practice of human resources. From online/e-recruiting to e-learning and the growing interest in data analytics, the practice of human resources has become technology centric. Given the overlap of human resource practice and information systems, both fields need to work together to develop models and theories that advance the practice of HRIS in organizations. Therefore, this special issue a) briefly reviews the history of the HRIS field, b) advances theory and research that stands at the intersection of HRIS and HCI, and c) suggest new directions for research at the intersection of HRIS and HCI.

Can we Take User Responses at Face Value? Exploring Users’ “Self-stated” and “Derived” Importance of Utilitarian versus Hedonic Software Features

Adarsh Kumar Kakar — Tue, 27 Dec 2016 11:46:57 PST

Empirical studies in the product development literature have shown that the users’ self-reported importance of product attributes differs from the derived importance of product attributes obtained through the attributes’ correlation with an external criterion such as user satisfaction. However, no study has examined this phenomenon in the context of software products. This investigation is important because the present-day software requirement-prioritization techniques are based on capturing users’ self-reported importance of new software product features. As such, I develop a method in the study to capture the derived user importance of new features. The findings show that the implicitly derived importance of software attributes differs from the importance rankings assigned to them using requirement prioritization techniques. Further, I found that the implicitly derived user importance to identify the determinants of user satisfaction more accurately than the prioritization techniques based on self-stated user importance. I discuss the implications of this promising new approach for practice and future research in requirements prioritization.

Limited Information and Quick Decisions: Consumer Privacy Calculus for Mobile Applications

Mark J. Keith et al. — Sun, 02 Oct 2016 22:21:58 PDT

Mobile applications (also known as “apps”) have rapidly grown into a multibillion-dollar industry. Because they are available through devices that are “always on” and often with the user, users often adopt mobile apps “on the fly” as they need them. As a result, users often base their adoption and disclosure decisions only on the information provided through the mobile app delivery platform (e.g., the Apple App Store™ or Google Play™). The fact that using a mobile app often requires one to disclose an unprecedented combination of personal information (e.g., location data, preferences, contacts, calendars, browsing history, music library) means that one makes a complex risk/benefit tradeoff decision based on only the small amount of information that the mobile app delivery platform provides—and all in a short period of time. Hence, this process is much shorter and much riskier than traditional software adoption. Through two experiments involving 1,588 mobile app users, we manipulated three primary sources of information provided by a platform (app quality ratings, network size, and privacy assurances) to understand their effect on perceptions of privacy risks and benefits and, in turn, how they influence consumer adoption intentions and willingness to pay (WTP). We found that network size influenced not only perceived benefits but also the perceived risks of apps in the absence of perfect information. In addition, we found that integrating a third party privacy assurance system into the app platform had a significant influence on app adoption and information disclosure. We also found that a larger network size reduces LBS privacy risk perceptions, which confirms our information cascade hypothesis. We discuss the implications of these findings for research and practice.

The Happiness Premium: The Impact of Emotion on Individuals’ Willingness to Pay in Online Auctions

Lingyao Yuan et al. — Thu, 29 Sep 2016 19:36:54 PDT

Much research across various disciplines has studied individuals’ bidding behavior in online auctions. Emotion is an important factor affecting individual behavior, but we know little about its effects in online auctions. We conducted a lab experiment to investigate the impact of positive emotion on individuals’ willingness to pay in online auctions. We found that individuals with positive emotions bid more than those with neutral emotions; that is, they paid a “happiness premium” of about 10 percent. The effect size was medium (Cohen’s d = 0.51). This study contributes to electronic commerce literature by identifying emotion as an important factor affecting online auction behavior. The findings also provide guidance to auction website design: websites can increase bid amounts by inducing positive emotions in potential customers.

Impact of Human Resource Information System Policies on Privacy

Kimberly M. Lukaszewski et al. — Sun, 26 Jun 2016 10:41:49 PDT

Organizations are increasingly using human resource information systems (HRIS) to collect and store employee data to enhance employment decision making. In this paper, using a 2 x 2 x 2 experimental design, we 1) examine the effects of three HRIS policies on employees’ perceptions of invasion of privacy, 2) assess the moderating effects of amount of work experience on the relations between these HRIS policies and employees’ perceptions of invasion of privacy and 3) discuss the implications of these findings for developing fair information policies. Results revealed that individuals perceived a HRIS was more of an invasion of privacy when HRIS data were used for only the benefit of the organization than when it was used to benefit employees. In addition, the results indicated that individuals perceived that a HRIS was more invasive of privacy when the data were accessed by supervisors than when they were accessed by the HR department only. Furthermore, individuals' amount of work experience moderated the relations between (a) purpose of the data collection, and (b) access to data and perceptions of invasion of privacy. Implications for theory, research, and practice are discussed.

Website Credibility Assessment: An Empirical - Investigation of Prominence-interpretation Theory

Joey F. George et al. — Sun, 26 Jun 2016 10:41:45 PDT

While a variety of research studies have examined factors that influence individuals’ attitudes toward and use of websites, an important yet understudied stream looks at the role of website credibility. We examine website credibility through the lens of prominence-interpretation theory. Fogg (2003) developed this theory to help explain the relationships between what users observe about a website, how they interpret it, and how observation and interpretation together determine website credibility. In this paper, we look specifically at the relationship between prominence and interpretation and how these variables interact to influence attitudes about website credibility. We examined this relationship using a controlled laboratory experiment in which we exposed subjects to a website and asked them what they saw and how they interpreted what they saw. We analyzed these data using discriminant analysis and show that the interaction between prominence and interpretation accurately predicts attitudes about credibility, which offers strong support for prominence-interpretation theory. We discuss these findings and their implications for theory and practice.

Exploring Failure and Engagement in a Complex Digital Training Game: A Multi-method Examination

Matthew L. Jensen et al. — Mon, 28 Mar 2016 09:06:55 PDT

Digital games are ideal for training complex decision making skills because they allow players to experience decision making processes and consequences. However, training complex skills often results in failure, which may undermine learning engagement. Traditional training methods employing observational learning (e.g., training videos) do not cause learners to fail but forfeit experiential learning that makes training games so engaging. Our exploratory work addresses the trade-off between experiencing and observing failure and explores their effect on the level of training engagement. Building on past engagement research, we argue that learning engagement contains both cognitive and affective facets and that these facets may diverge, especially when individuals experience failure. To test these ideas, we conducted an experiment (N = 156) comparing engagement in game-based training, in which participants experienced failure, and video-based training, in which participants observed failure. We collected cognitive and affective indicators of engagement using physiological and self-report measures. We found game-based experiential learning increased such indicators of engagement as attention and temporal disassociation even though players widely failed to meet game objectives. Players also experienced elevated arousal and decreased positive affect. In addition, we compared physiological measures of engagement with self-reported measures and discuss their merits and limitations.

Designing Interfaces for Older Users: Effects of Icon Detail and Semantic Distance

Netta Ganor et al. — Mon, 28 Mar 2016 09:06:47 PDT

Older users are increasingly using digital means (especially the Internet and mobile devices) for work and leisure. Yet, until recently, researchers have not paid sufficient attention to how one can adapt the human-computer interface to older users. In this paper, we propose an approach to adapting interfaces to older users that is based on the knowledge of design and knowledge of age-related capabilities and needs. We concentrate on icons, which are perhaps the most common means of controlling human-computer interaction. First, we determined age-related cognitive impairments that might affect icon identification and selected two icon attributes that we could adapt to overcome performance degradation. We then conducted an experiment to test the hypothesized differences between young and old adults in terms of the impact of icon design on user performance. The two attributes of icon design were level of detail in the icon (i.e., the number of elements that constitute the icon) and its semantic distance (i.e., the distance between the meaning of the icon and function it represents). We found that both attributes affected the performance of older users more strongly than they did young users except for an extreme case in which the negative impact on young users when adapting for older users was devastating. We believe that these results demonstrate that adaptation of icons for older users is desirable and feasible but must be done with caution. We need more research to determine the nuances and limitations of adaptation to understand the adaptation of other design attributes by going beyond the cognitive aspects to the physical, affective, and social aspects of human computer interaction.

Unraveling the Mystery of New Technology Use: An Investigation into the Interplay of Desire for Control, Computer Self-efficacy, and Personal Innovativeness

Sharen Bakke et al. — Wed, 30 Dec 2015 14:11:46 PST

In this paper, we examine how intrinsically motivated competence and autonomy (two basic psychological needs derived from self-determination theory) in concert with personal innovativeness in IT motivate individuals to try new information technologies. In a study with 202 participants we found 1) competence, operationalized as general computer self-efficacy (GCSE), and 2) autonomy, operationalized as desire for control over information technology (DCIT), to positively influence individuals’ intention to use new or unfamiliar technologies. Further, we hypothesize and find evidence of an interaction effect that suggests there may be a tradeoff between these constructs. That is, individuals may be inclined to use new technologies because they perceive themselves as having high levels of ability or because they have high levels of desire; they are either competence-driven or desire-driven users. Therefore, correctly identifying potential users into the appropriate user category and providing the necessary training or control mechanisms will likely increase an individual’s intention to try new and innovative IT products.

Instructor versus Peer Attention Guidance in Online Learning Conversations

Evren Eryilmaz et al. — Wed, 30 Dec 2015 14:11:44 PST

This paper reports a theory-driven experimental study for designing and evaluating two different forms of attention-guidance functionalities integrated into an anchored-discussion system. Using social constructivism as a motivating theory, we constructed a theoretical framework that emphasizes the importance of students’ attention allocation in online learning conversations and its influence on message quality and interaction patterns. The development of the functionalities, named faded instructor-led and peer-oriented attention guidance, aimed to direct students’ attention toward instructional materials’ central domain principles while offering them an open learning environment in which they could choose their own topics and express their own ideas. We evaluated the functionalities with heat map analysis, repeated measures general linear model analysis, and sequence analysis to assess the utility of the developed functionalities. Results show that attention guidance helped students more properly allocate their attention in online learning conversations. Furthermore, we found that the improved attention allocation led to better quality of students’ online learning conversations. We discuss implications for researchers and practitioners who wish to promote more fruitful online discussions.