Paper Type
Complete
Abstract
Low-resource languages face challenges due to limited linguistic resources. In 2024, Aya, a multilingual model supporting 101 languages, was introduced. This study evaluates Aya's performance and the efficacy of a few-shot learning approach in Aspect-Based Sentiment Analysis, Hate Speech Detection, Irony Detection, and Question-Answering using ABSAPT 2022, ToLD-BR, IDPT 2021, and SQUAD v1.1 datasets. Without fine-tuning, Aya demonstrated strong results in QA, achieving a 58.79% Exact Match score, surpassing Portuguese-specific models. However, it struggled in Hate Speech Detection, with an F1-score of 0.64, well below Sabiá-7B's 0.94. ABSA performance improved without neutral examples, but the model faced challenges with complex slang and context-dependent features. These findings highlight Aya's potential in multilingual NLP while demonstrating the capabilities and limitations of few-shot learning as an evaluation strategy for LLMs in low-resource scenarios.
Paper Number
2289
Recommended Citation
Carvalho, Eduarda Abreu; Lopes, Émerson; da Rocha Junqueira, Júlia; Correa, Ulisses B.; and Freitas, Larissa, "Aya in Action: An Investigation of its Abilites in Aspect-Based Sentiment Analysis, Hate Speech Detection, Irony Detection and Question-Answering" (2025). AMCIS 2025 Proceedings. 3.
https://aisel.aisnet.org/amcis2025/lacais/lacais/3
Aya in Action: An Investigation of its Abilites in Aspect-Based Sentiment Analysis, Hate Speech Detection, Irony Detection and Question-Answering
Low-resource languages face challenges due to limited linguistic resources. In 2024, Aya, a multilingual model supporting 101 languages, was introduced. This study evaluates Aya's performance and the efficacy of a few-shot learning approach in Aspect-Based Sentiment Analysis, Hate Speech Detection, Irony Detection, and Question-Answering using ABSAPT 2022, ToLD-BR, IDPT 2021, and SQUAD v1.1 datasets. Without fine-tuning, Aya demonstrated strong results in QA, achieving a 58.79% Exact Match score, surpassing Portuguese-specific models. However, it struggled in Hate Speech Detection, with an F1-score of 0.64, well below Sabiá-7B's 0.94. ABSA performance improved without neutral examples, but the model faced challenges with complex slang and context-dependent features. These findings highlight Aya's potential in multilingual NLP while demonstrating the capabilities and limitations of few-shot learning as an evaluation strategy for LLMs in low-resource scenarios.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
LACAIS