Abstract

With the proliferation of open innovation, User-Generated Content (UGC) on social media has become a critical source for identifying latent user needs and market opportunities. However, extracting scarce innovation signals from massive, unstructured, and noisy text data remains a significant challenge. Traditional anomaly detection methods, such as standard autoencoders, often rely on global statistical deviations and fail to capture the subtle semantic uniqueness associated with innovative ideas. To address this limitation, this study proposes an improved deep autoencoder model integrated with a Multi-Head Self-Attention mechanism. By dynamically assigning weights to semantic segments, the model enhances the detection of unique user insights while suppressing irrelevant noise. We validated the model using an empirical dataset of 8635 user reviews on VR glasses from mainstream Chinese social platforms. The experimental results demonstrate that the proposed model significantly outperforms traditional methods, reducing the initial screening candidate pool by 47.8% while maintaining high sensitivity. Through an iterative screening strategy, the dataset volume was reduced by 99.53%, isolating 41 core reviews as profound semantic anomalies. Subsequent expert validation confirmed that these effectively captured high-value’ lead user’ innovation ideas. This research contributes a cost-effective, automated “pre-filtering” approach for innovation mining, enabling enterprises to efficiently discover lead user needs from the “ocean of noise.”

Download

Share

COinS