Location
Hilton Waikoloa Village, Hawaii
Event Website
https://hicss.hawaii.edu/
Start Date
7-1-2025 12:00 AM
End Date
10-1-2025 12:00 AM
Description
The vast volume of chat messages generated during esports events on Twitch represents a valuable source of data for understanding audience behavior. However, the sheer quantity and dynamic nature of this data make manual analysis impractical. This study addresses this challenge by introducing FinTwitchBERT, a model fine-tuned to classify Twitch chat messages into four categories based on their uniqueness. Our model demonstrates the ability to distinguish between original content, repetitive messages such as emote spamming, formulaic messages, and interactive commands chat participants use to interact with channel bots. Pre-trained on over 18 million Finnish Twitch chat messages and utilizing a combination of semi-supervised learning and iterative pseudo-labeling with human-in-the-loop validation, FinTwitchBERT achieves 97.42% accuracy on a test set of unseen chat messages with a limited initial dataset of only 7,529 manually annotated messages.
Recommended Citation
Lindroos, Jari; Peltonen, Jaakko; Välisalo, Tanja; Koskimaa, Raine; and Toivanen, Ida, "From PogChamps to Insights: Detecting Original Content in Twitch Chat" (2025). Hawaii International Conference on System Sciences 2025 (HICSS-58). 10.
https://aisel.aisnet.org/hicss-58/dsm/data_analytics/10
From PogChamps to Insights: Detecting Original Content in Twitch Chat
Hilton Waikoloa Village, Hawaii
The vast volume of chat messages generated during esports events on Twitch represents a valuable source of data for understanding audience behavior. However, the sheer quantity and dynamic nature of this data make manual analysis impractical. This study addresses this challenge by introducing FinTwitchBERT, a model fine-tuned to classify Twitch chat messages into four categories based on their uniqueness. Our model demonstrates the ability to distinguish between original content, repetitive messages such as emote spamming, formulaic messages, and interactive commands chat participants use to interact with channel bots. Pre-trained on over 18 million Finnish Twitch chat messages and utilizing a combination of semi-supervised learning and iterative pseudo-labeling with human-in-the-loop validation, FinTwitchBERT achieves 97.42% accuracy on a test set of unseen chat messages with a limited initial dataset of only 7,529 manually annotated messages.
https://aisel.aisnet.org/hicss-58/dsm/data_analytics/10