Abstract
AI-enabled information systems often operate on nonstationary data streams, where changes in data distribution may reduce the quality of automated decisions. This study presents a compact chunk-wise evaluation framework for comparing unsupervised and error-based concept drift detectors. The study focuses on two feature-space detectors, Walsh–Hadamard Drift Detector (WHDD) and One-Class Drift Detector (OCDD), and three error-based baselines: ADWIN, DDM, and EDDM. The detectors are evaluated on controlled synthetic streams with abrupt and gradual drifts. Besides the commonly used distance-based metrics D1 and D2, we include the count-consistency index R, which captures excessive or insufficient alarm generation. The results show that low D2 values alone may be misleading, because oversensitive detectors can cover true drift locations while producing unstable alarm patterns. WHDD provides a more balanced profile, combining competitive drift coverage with better alarm-count consistency. The study demonstrates that reliable detector assessment should jointly consider alarm localization, drift coverage, and alarm stability.
Paper Type
Poster
DOI
10.62036/ISD.2026.51
Chunk-Wise Evaluation of Unsupervised and Error-Based Concept Drift Detectors for AI-Enabled Information Systems
AI-enabled information systems often operate on nonstationary data streams, where changes in data distribution may reduce the quality of automated decisions. This study presents a compact chunk-wise evaluation framework for comparing unsupervised and error-based concept drift detectors. The study focuses on two feature-space detectors, Walsh–Hadamard Drift Detector (WHDD) and One-Class Drift Detector (OCDD), and three error-based baselines: ADWIN, DDM, and EDDM. The detectors are evaluated on controlled synthetic streams with abrupt and gradual drifts. Besides the commonly used distance-based metrics D1 and D2, we include the count-consistency index R, which captures excessive or insufficient alarm generation. The results show that low D2 values alone may be misleading, because oversensitive detectors can cover true drift locations while producing unstable alarm patterns. WHDD provides a more balanced profile, combining competitive drift coverage with better alarm-count consistency. The study demonstrates that reliable detector assessment should jointly consider alarm localization, drift coverage, and alarm stability.
Recommended Citation
Porwik, P. & Mensah Dadzie, B.(2026). Chunk-Wise Evaluation of Unsupervised and Error-Based Concept Drift Detectors for AI-Enabled Information Systems. In M. Valenta, B. Mannová, R. Pergl, A. Przybylek, M. Lang, H. Linger, C. Schneider, N. Iivari, & E. Insfran (Eds.), Making ISD Sustainable: Reloaded with AI and Automation (ISD2026 Proceedings). Prague, Czech Republic: Czech Technical University in Prague. ISBN: 978-80-01-07585-2. https://doi.org/10.62036/ISD.2026.51