Determining the reliability of KBS has become an important research area due to the application of KBSs to areas where misplaced confidence can cause large monetary losses or even loss of life. Developers need a set of criteria to evaluate KBS’s reliability, i.e. reliability criteria. Using arguments from philosophy of science, we define three criteria, thus providing a theoretical basis for judging the reliability of a KBS. Previous researchers have argued for or against specific criteria, however there is little agreement on definitions and groupings of these criteria (Nazareth and Kennedy 1993). The lack of agreement is, in part, due to the lack of a theoretical foundation underpinning KBS reliability. A theoretical viewpoint provides a basis for understanding and generalizing results. Without a theory to explain results, findings may be idiosyncratic to a particular system. Worse, findings that may not be idiosyncratic are easier to dismiss as they lack a basis for generalizability.
Shaft, Teresa and Gamble, Rose, "A Theoretical Basis for the Assessment of Rule-Based System Reliability" (1999). AMCIS 1999 Proceedings. 28.