Abstract
Predicting the health of software projects is a critical challenge in Information Systems Development (ISD). This paper presents an automated nine-stage pipeline for project health classification that integrates static code metrics with behavioral repository analytics collected via the GitHub API. A labeled dataset of 777 open-source repositories across eight language groups is constructed from 2,113 candidates on a realistically imbalanced class distribution (37% healthy / 63% at-risk), with rule-based labeling applied with explicit leakage mitigation through ablation study. Four ML classifiers are evaluated under 5-fold cross-validation with SHAP explainability and McNemar statistical testing. XGBoost achieves F1 macro = 0.947 ± 0.016 and AUROC = 0.984, significantly outperforming Logistic Regression (p = 0.0002) and SVM (p < 0.0001). SHAP analysis reveals that behavioral metrics dominate static code metrics as health predictors by a ratio of 106:1. The model generalises across all eight language groups (F1: 0.916–0.979), enabling explainable, AI-driven decision support for IS project portfolio management and proactive governance.
Paper Type
Short Paper
DOI
10.62036/ISD.2026.99
Software Project Health Prediction in ISD: Integrating Static Code Metrics with AI-Based Repository Analytics
Predicting the health of software projects is a critical challenge in Information Systems Development (ISD). This paper presents an automated nine-stage pipeline for project health classification that integrates static code metrics with behavioral repository analytics collected via the GitHub API. A labeled dataset of 777 open-source repositories across eight language groups is constructed from 2,113 candidates on a realistically imbalanced class distribution (37% healthy / 63% at-risk), with rule-based labeling applied with explicit leakage mitigation through ablation study. Four ML classifiers are evaluated under 5-fold cross-validation with SHAP explainability and McNemar statistical testing. XGBoost achieves F1 macro = 0.947 ± 0.016 and AUROC = 0.984, significantly outperforming Logistic Regression (p = 0.0002) and SVM (p < 0.0001). SHAP analysis reveals that behavioral metrics dominate static code metrics as health predictors by a ratio of 106:1. The model generalises across all eight language groups (F1: 0.916–0.979), enabling explainable, AI-driven decision support for IS project portfolio management and proactive governance.
Recommended Citation
Bykowski, P.A.(2026). Software Project Health Prediction in ISD: Integrating Static Code Metrics with AI-Based Repository Analytics. In M. Valenta, B. Mannová, R. Pergl, A. Przybylek, M. Lang, H. Linger, C. Schneider, N. Iivari, & E. Insfran (Eds.), Making ISD Sustainable: Reloaded with AI and Automation (ISD2026 Proceedings). Prague, Czech Republic: Czech Technical University in Prague. ISBN: 978-80-01-07585-2. https://doi.org/10.62036/ISD.2026.99