Abstract

AI systems are increasingly deployed in advisory, decision support, and guidance contexts, yet in psychologically grounded applications such as well-being support and developmental interventions, their interpretability is critical and often inadequate. Standard RAG architectures address hallucination risk by anchoring outputs in external documents, but rely on vector-based semantic similarity. This distinction matters because users do not merely need relevant-sounding information; they need solutions that impact the underlying problem. From an IS perspective, decision support systems must align their logic with users' cognitive models and domain understanding, a standard embedding-based retrieval alone cannot meet. This talk presents a design science-informed approach to constructing and integrating a domain-specific knowledge graph for positive psychology into a RAG architecture to enable graph-aware retrieval and improved reasoning traceability. We treat the KG-RAG pipeline as an IS artifact targeting a clearly scoped problem: the absence of structured, theory-aligned knowledge representation in AI guidance systems for psychological support. Using NVIDIA's txt2kg framework, a Llama 3.1 (8B) inference engine extracts (subject, predicate, object) triples from sentence-level document chunks, stored as labeled edges in ArangoDB and retrieved via multi-hop graph traversal, deriving relationships between constructs beyond semantic similarity. To construct a prototype of positive psychology knowledge graph, bibliographic records from OpenAlex were filtered to 46 topic categories with at least 75 records, yielding 8,245 unique records and a final graph of 4,273 nodes and 3,580 relationships. We evaluate graph-based retrieval using an LLM-as-a-judge scoring pipeline, where a Llama 3.1 model was prompted to score the correctness of a predicted answer against a ground-truth response on a 0–1.0 scale. Averaged across all 46 positive psychology categories, the KG-RAG system achieves a mean score of 0.61, indicating outputs capture approximately 61% of ground-truth semantic content. Highest-performing categories include Entrepreneurship Studies (M = 0.88), Behavioral Health and Interventions (M = 0.73), and Healthcare Professionals' Stress and Burnout (M = 0.71). We outline how this knowledge graph can be integrated into TrueNorth, which is an existing multi-agent RAG system for early-career STEM professional mentoring grounded in the PERMA+4 framework for workplace well-being. to. Where the current TrueNorth architecture retrieves via vector embeddings, KG-augmented retrieval would expose structured relational paths linking constructs, interventions, and publications, enabling users to trace generated guidance back to underlying psychological theory. This talk invites IS community feedback on two threads we are actively working to strengthen: the design science framing of the KG-RAG pipeline as an explainability artifact, and the theoretical grounding of explainability as an emergent design property in psychology-aligned AI systems. We welcome discussion on evaluation strategies, applicable IS theory, and collaborators interested in extending this work to other high-stakes advisory domains including education, healthcare, and professional development.

Download

Share

COinS