AIS Transactions on Replication Research


This paper provides the gold standard sets described and used by Palese and Piccoli (2020) which advance a scalable method, Human Interpretable Topics (HIT), for assessing the interpretability of topic modeling results. Gold standard set are a small collection of documents extracted from a dataset and manually labeled by humans. The role of such a gold standard dataset is to establish a “ground truth” against which researchers can benchmark the results generated by algorithmic topic models. We first provide a detailed description of the classification procedure used to create the labeled sets. Then, we make available descriptive statistics of the three different gold standard sets named respectively: inclusive, full agreement and partial agreement. These gold standard sets can be used to benchmark different/new models built in research analyzing online customers’ reviews in the context of the lodging industry. We hope a larger number of researchers will follow our example and use the AIS Transactions on Replication Research journal to share open access gold standard sets in different areas of interest.