Paper Number
ICIS2025-2124
Paper Type
Short
Abstract
Leveraging the structural similarities between artificial neural networks and human visual processing, this study proposes a novel approach to approximate hierarchical visual complexity perception with the layered attention matrices of transformer-based models. Using the vision encoder of CLIP, a transformer-based multimodal model, we propose Hierarchical Attention Entropy (HAE) to simulate stagewise brain activation during visual perception. Our initial results from a social media dataset reveal that early-stage attention entropy (low-level and high-level AE) positively correlates with user engagement, while late-stage entropy (high-level AE) shows a negative relationship. This method contributes a cognitively inspired data augmentation approach and offers new insights into how hierarchical visual processing influences user behavior online.
Recommended Citation
Cai, Jingyuan and Wang, Chong (Alex), "Revealing Visual Cognition with AI Simulator: Hierarchical Attention Entropy Derived from Artificial Neural Network" (2025). ICIS 2025 Proceedings. 14.
https://aisel.aisnet.org/icis2025/is_researchmethods/is_researchmethods/14
Revealing Visual Cognition with AI Simulator: Hierarchical Attention Entropy Derived from Artificial Neural Network
Leveraging the structural similarities between artificial neural networks and human visual processing, this study proposes a novel approach to approximate hierarchical visual complexity perception with the layered attention matrices of transformer-based models. Using the vision encoder of CLIP, a transformer-based multimodal model, we propose Hierarchical Attention Entropy (HAE) to simulate stagewise brain activation during visual perception. Our initial results from a social media dataset reveal that early-stage attention entropy (low-level and high-level AE) positively correlates with user engagement, while late-stage entropy (high-level AE) shows a negative relationship. This method contributes a cognitively inspired data augmentation approach and offers new insights into how hierarchical visual processing influences user behavior online.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
25-Research