Abstract

The Automated Compliance Engine (ACE) is an AI-orchestrated intrusion detection and compliance mapping engine that is designed and implemented to bridge the gap between low-level network security events and high-level regulatory requirements. ACE combines Suricata IDS (Intrusion Detection System) with a five-stage LangGraph pipeline to automatically parse network alerts, leverage a Retrieval-Augmented Generation (RAG) architecture to map threats to NIST SP 800-53 Rev. 5 security controls, and deploy generated rules to a target system via SSH. The system is built on a two-virtual-machine VirtualBox environment, that includes a Kali Linux orchestrator and a Windows IDS host. It shows that AI-assisted automation can replace the manual work analysts usually do to move from detection to compliance mapping and countermeasure deployment. ACE is tested in eight simulated attack scenarios, including reconnaissance, web application attacks, DNS abuse, and denial-of-service attacks. The system found 100% alerts and zero false positives across three tests of legitimate traffic. For all the eight alerts processed, three resulted in deployable indicator-specific auto rules (37.5% rule-generation rate). All six rules generated are confirmed to have been deployed. The average confidence score for mapping NIST SP 800-53 across all eight alerts is 0.87. DeepEval LLM (Large Language Model) quality assessment (Challita, B., & Parrend, P., 2025) provides scores of 1.0 for Faithfulness and Answer Relevancy. The Hallucination score of 0.67 shows a known LLM-as-judge methodological issue, not an implementation flaw in the pipeline. This is supported by independent benchmarking of RAG evaluation frameworks under abstention policies (Siagian, L., 2025). The average time the pipeline took to run was about 173 seconds. SID (Security Identifier) based deduplication ensures that no unnecessary rules are used in pipeline runs, making the system idempotent and well-suited for long-running production environments.

Download

Share

COinS