The demand for measuring IT security performance is driven by regulatory, financial, and organizational factors. While several best practice metrics have been suggested, we observe a lack of consistent requirements against which IT security metrics can be evaluated. We address this research gap by adopting a methodological approach that is based on argumentation theory and an accompanying literature review. As a result, we derive five key requirements: IT security metrics should be (a) bounded, (b) metrically scaled, (c) reliable, valid and objective, (d) context-specific and (e) computed automatically. We illustrate and discuss the context-specific instantiation of requirements by using the practically used "vulnerability scanning coverage" and "mean-time-to-incident discovery" metrics as examples. Finally we summarize further implications of each requirement.
Yasasin, Emrah and Schryen, Guido, "Requirements for IT Security Metrics - an Argumentation Theory Based Approach" (2015). ECIS 2015 Completed Research Papers. Paper 208.