Abstract

Users' satisfaction with their information systems is generally recognized as one of the most important indicators of success in designing and implementing computer-based systms. User information satisfaction (UIS) is defined as "the extent to which users believe the information systems available to them meets their information requirements." [Ives, Olson, and Baroudi, 1983]The construct is often used as a surrogate for relative information system value because it is more accurately and easily measurable. [Nolan and Seward, 1974] Powers and Dickson [1973] concluded that "user satisfaction is the most critical criterion for measuring computer systems success or failure." User information satisfaction has been measured in several ways. Different instruments have been created by Larcker and Lessig [1980], Jenkins and Ricketts [1979], and Gallagher [ 1974]. Recently, Bailey and Pearson [1983] have developed an instrument for UIS that represents an important achievement in the evolution of this construct. Their instrument is broadly based and includes measures of both the system and support quality perceived by the user. Some evidence of the reliability and validity of the instrument has been provided. 77:e Bailey-Pearson U/S Instrument Bailey and Pearson (BP) reviewed 22 MIS studies for " factors affecting computer user satisfaction" and identified 36 constructs. Three further constructs were added to the list and it was tested for completeness using taped interviews with 32 middle managers. The researchers concluded that there was "a 0.99 probability that a mentioned (construct) was on the list (of 39)" [Bailey and Pearson, 1983, p. 532]. The thirty-nine constructs are listed in Table I. For each construct, four semantic differential seven point scales were developed and a fifth scale, common to all, asked about the importance of the construct to the respondent. A measure of the response to a construct by a respondent was estimated as the average of responses to the four semantic differential scales for the construct. The general measure of user information satisfaction for a respondent was estimated as the average of the construct estimates weighted by the importance measure of that construct, after having eliminated those for which the response on each of the four semantic differential scales were neutral: Using the instrument, data was gathered from 29 of the 32 managers interviewed during the development of constructs. Analysis of the data led the authors to conclude that the instrument was reasonably reliable and valid. The test sample was quite small and potentially biased since respondents had previously participated in developing the instrument. 1ves, Olson, and Baroudi [ 1983] chose to further study the empirical properties of the Bailey. Pearson (B-P) instrument with an additional objective to develop a shorter version of the instrument.

71:e /ves, Olson, and Baroudi Short Instrument: Ives, Olson, and Baroudi (IOB) collected data from 200 production managers (25 % response rate) in U. S. manufacturing organizations using the Bailey-Pearson instrument and a separate 4-scale instrument of information satisfaction. They performed an exploratory factor analysis on construct scores estimated from the BP data and obtained four significant factors or dimensions, as shown in Table I. Any construct that did not load on one or more factors at the 0.5 level or above was eliminated from a shortened version of the Bailey-Pearson instrument. As well, "vendor support" which loaded on its own factor was eliminated. Twenty-one of the original 39 constructs passed this test and were included in the shortened instrument. For each of these, two semantic differential scales were retained as well as the importance scale. The results of the factor analysis suggest that a multidimensional structure may underly the user information satisfaction construct. IOB discovered a three factor model of UIS (putting aside vendor support). The factors are: (1) EDP staff and services; (2) the quality of the information product; and (3) the knowledge and involvement levels of the user. Their instrument contains twenty-one questions, each with two semantic differential scales, that measure the three independent factors, but no general measures of user information satisfaction. Their method of arriving at this questionnaire eliminated any general measures of UIS since these measures, by definition, load across all factors. Since this was an exploratory analysis, no evidence was provided as to whether the measures of the underlying dimensional factors achieved convergent or discriminant validity. Evidence was provided that the shortened instrument is a reliable and valid measure ofuser information satisfaction. The 2-scale reliability scores were approximately equal to the 4-scale scores, supporting the claim of high reliability. The general measure of UIS estimated using 21 constructs and the measure using all 39 constructs had a correlation across respondents of 0.9. The correlations of each of these measures with the measure estimated using the separate 4-scale instrument were 0.54 and 0.55 respectively. This evidence supports the claim that the shortened instrument is measuring approximately the same thing as the complete Bailey-Pearson instrument. There are several problems with the evidence provided to support the reliability and validity of the shortened instrument. The reliability estimates obtained are unusually high for this type of instrument, ranging between 0.81 and 0.97 respectively. This evidence supports the claim that the shortened instrument is measuring approximately the same thing as the complete Bailey-Pearson instrument. There are several problems with the evidence provided to support the reliability and validity of the shortened instrument. The reliability estimates obtained are unusually high for this type of instrument, ranging between 0.81 and 0.97. This may be explained in part by a mechanical methods bias introduced by placing all four semantic differential scales on the same page and scored in the same direction. Thus, the analysis of 4-scale, versus 2- scale reliabilities may not be accurate. As well, no estimate was provided of the reliability with which the 21 questions measure the underlying UIS variable. The 0.9 correlation between the 39-construct instrument and the 21-construct instrument may partly be explained by the fact that the two instruments have 21 constructs in common. Just as this correlation is significantly different from zero, it is also significantly different from 1.0 (significant at the 0.001 level). This may be a better test of whether the shortened instrument is significantly different from the Bailey-Pearson instrument. Evidence tha tthe two instruments correlate equally with a third 4-scale instrument is of little help. There is no evidence that the third instrument is either a reliable or valid measure ofuser information satisfaction. Central to the process by which the researchers eliminated constructs from the instrument was an exploratory factor analysis. As they observed, "the ratio of sample size to number of scales in the study (7:1) must also be regarded with some caution. ." [Ives, Olson, Baroudi, 1983, p. 789]. Thus, one place to begin an analysis of the shortened version of the Bailey-Pearson instrument is by employing the identified factor structure to test the measurement qualities of the instrument. Successful results on this test will support the hypothesized multi-dimensional structure of user information satisfaction. The factor structure obtained by Ives, Olson, and Baroudi is significant not only because they employed it to make decisions about which constructs to eliminate from the shortened instrument. Perhaps more significantly, the factor analysis has yielded underlying dimensions to user information satisfaction that point toward a causal model of UIS. Such a model coutdbe usedas an importantdiagnostictool for information systems builders. Such an important potential development deserves careful study.

Download

Share

COinS