Abstract
Deep-learning pipelines for sonar-based UXO classification increasingly rely on synthetic data generated in Digital Twin environments. While such pipelines support scalable development of AI-enabled decision-support services, they also risk introducing hidden information leakage, causing models to learn non-physical shortcuts instead of target-relevant acoustic cues. In this paper, we consider a binary classification task (UXO vs.\ non-UXO) and present a five-stage protocol for auditing the integrity of synthetic sonar datasets before further model development and deployment. The protocol combines complexity stress-testing, saliency/Grad-CAM inspection, object masking, and bias-only training on masked images. In our case study, it revealed contamination linked to the simulator data-generation procedure. A leaky dataset achieved near-perfect balanced accuracy (up to 99.22\%), whereas the corrected dataset produced lower but more plausible results (97.35\% on an idealized seabed and 83.62\% on a realistic seabed). The protocol can therefore serve as a practical quality-assurance component for synthetic-data pipelines.
Paper Type
Short Paper
DOI
10.62036/ISD.2026.89
Protocol for Detecting Synthetic Artifacts: A Case Study on UXO Sonar Imagery
Deep-learning pipelines for sonar-based UXO classification increasingly rely on synthetic data generated in Digital Twin environments. While such pipelines support scalable development of AI-enabled decision-support services, they also risk introducing hidden information leakage, causing models to learn non-physical shortcuts instead of target-relevant acoustic cues. In this paper, we consider a binary classification task (UXO vs.\ non-UXO) and present a five-stage protocol for auditing the integrity of synthetic sonar datasets before further model development and deployment. The protocol combines complexity stress-testing, saliency/Grad-CAM inspection, object masking, and bias-only training on masked images. In our case study, it revealed contamination linked to the simulator data-generation procedure. A leaky dataset achieved near-perfect balanced accuracy (up to 99.22\%), whereas the corrected dataset produced lower but more plausible results (97.35\% on an idealized seabed and 83.62\% on a realistic seabed). The protocol can therefore serve as a practical quality-assurance component for synthetic-data pipelines.
Recommended Citation
Ściegienka, P. & Blachnik, M.(2026). Protocol for Detecting Synthetic Artifacts: A Case Study on UXO Sonar Imagery. In M. Valenta, B. Mannová, R. Pergl, A. Przybylek, M. Lang, H. Linger, C. Schneider, N. Iivari, & E. Insfran (Eds.), Making ISD Sustainable: Reloaded with AI and Automation (ISD2026 Proceedings). Prague, Czech Republic: Czech Technical University in Prague. ISBN: 978-80-01-07585-2. https://doi.org/10.62036/ISD.2026.89