Loading...
Paper Number
2621
Paper Type
Short
Abstract
Interest in Environmental Social Governance (ESG) and Sustainable Development Goals (SDG) has become paramount. Companies embed their ESG achievements, plans, and commitments into their annual reports in a non-standardized textual format. We propose a retrieval system for increasing transparency and comparability in ESG reporting, enabling various stakeholders to compare sustainability statements across annual reports. We employed qualitative and quantitative evaluation methods to select the most suitable embedding model for the proposed ESG retrieval application. Our results show that the domain-relatedness of models has no positive impact on the retrieval performance in the ESG domain. However, domain-related evaluation datasets support a more relevant approach for model evaluation in the domain of interest. We have developed a sustainable and efficient benchmark for ESG-related retrieval systems that is faster and more resource-efficient than generic benchmarks, which typically include many datasets that are not relevant to the domain at hand.
Recommended Citation
Winkel, Fabian; Geierhos, Michaela; and Fink, Josef, "Evaluating Embedding Models for Retrieving ESG Information from Annual Business Reports" (2024). ICIS 2024 Proceedings. 23.
https://aisel.aisnet.org/icis2024/aiinbus/aiinbus/23
Evaluating Embedding Models for Retrieving ESG Information from Annual Business Reports
Interest in Environmental Social Governance (ESG) and Sustainable Development Goals (SDG) has become paramount. Companies embed their ESG achievements, plans, and commitments into their annual reports in a non-standardized textual format. We propose a retrieval system for increasing transparency and comparability in ESG reporting, enabling various stakeholders to compare sustainability statements across annual reports. We employed qualitative and quantitative evaluation methods to select the most suitable embedding model for the proposed ESG retrieval application. Our results show that the domain-relatedness of models has no positive impact on the retrieval performance in the ESG domain. However, domain-related evaluation datasets support a more relevant approach for model evaluation in the domain of interest. We have developed a sustainable and efficient benchmark for ESG-related retrieval systems that is faster and more resource-efficient than generic benchmarks, which typically include many datasets that are not relevant to the domain at hand.
When commenting on articles, please be friendly, welcoming, respectful and abide by the AIS eLibrary Discussion Thread Code of Conduct posted here.
Comments
10-AI