Abstract

In this paper we describe a new tool for planning, creating and conducting wide-ranging usability data acquisition throughout the system life cycle from inception to replacement. This aids sub-culturally targetable, on-line mass-consultation applicable to usability studies and change management. Usability and web data intelligence mining is made possible by the system capturing data from users, asynchronously on a distributed network, with minimised annoyance and judgement distortion effects. Previous research has shown that human judgements, particularly retrospective as distinct from real-time evaluations of painful experiences, are fundamentally flawed when superseded by other experiences. In evaluation studies as in any knowledge elicitation exercise (whether for requirements specification, expert systems prototyping or IS impact analysis) it is vital that unarticulated or poorly articulated data is captured as completely as possible whilst minimising distortion bias effects and annoyance of the user. A monolithic data elicitation method often proves inadequate for requirements acquisition or usability data whereas a dynamic planning framework can provide the execution monitoring and contextually-aware control of the enquiry process, as prescribed in Llemex_rb (Badii 1986/87/88). Such meta-level reasoning needs meta-methodological knowledge of the situated applicability of methods so as to choose suitable techniques to capture user data. Such data can range from simple static IDs to highly dynamic data on underlying patterns of multi-modal user behaviour; with various life-cycle models, sub-languages, semiotics and dispositional attitudes (Badii 1986, 1999a,b,c). PopEval_MB as an unobtrusive, on-line, mass-personalisation tool replaces the traditional paper-based survey methods, which suffer from problems of usability data distortion and acquisition management. It serves an enquiry methodology, which is contextually sensitive to the capture problems of the particular data type(s) being targeted at any time thus guiding the selection of suitable elicitation techniques along the way (Badii 1986/96). The enquiry methodology itself is a sub-system within a meta-methodology of frameworks and tools for IS/IT impact analysis and IS cultural compatibility management. This meta-methodology is referred to as Cultural Accommodation Analysis with Sensitised Systems for User/Usability Relationships and Reachabilities Evaluation (C-Assure); under a research programme directed by P3i at UCN (Badii et al 1996, 1999a,b,c). This paper describes the motivation for C-Assure in researching applicable meta-models that minimise the risks in IS development and adoption. It gives an overview of the tools that C-Assure seeks to incorporate into an integrated IS Planning, Development and Diffusion Support Environment (IPDSE) of which the tools for usability evaluation, mass-personalisation and web intelligence ie PopEval_MB and Web_Eval_AB are the focus of this work. The paper explains the theoretical foundations and the hypotheses to be tested in terms of the human Judgement and Decision Making, and, the Pleasure and Pain Recall, or (J/DM)-PPR theoretic effects which also motivated the design of PopEval_MB. Our results support the recent findings from cognitive psychology studies in applying the research on Pleasure and Pain Recall (PPR) to Human Computer Interaction (HCI). In this context we have validated the influence of factors modifying J/DM; specifically the effects of 'duration neglect' and 'peak-and-end evaluations’. Thus the empirical studies, as performed here, have provided the first supportive evidence for the J/DM and PPR results from earlier research in psychology as can be applied to the fields of software engineering; in particular to Web Mediated Systems (WMS) for on-line shopping as an exemplar. We maintain that more expressive causal models of usability are needed for the increasingly more volatile user environments of emergent interactive systems such as WMS. We propose a new definition and a process model for dynamic usability, distinguishing instantaneous and steady state usability. The results indicate that PopEval_MB and WebEval_AB deliver their intended functionality with minimal user annoyance and distortion bias. We show how PopEval_MB can be used to by-pass, interpret and exploit natural J/DM-PPR biases; to enable the elicitation of least-distorted usability data intelligence; to reveal the precise root causes of, and the routes to, perceived user (dis)satisfaction. This study also confirms the validity of our new dynamic usability process model, which exploits the natural J/DM–PPR saliencyrecency effects and is thus more relevant to the emergent click-happy WMS user environments. The results can be exploited in interpretivist-iterative approaches to IS deployment, diffusion and change management, enterprise health analysis, marketing, design of WebAds and culturally inter-operable systems generally

Download

Share

COinS