The proliferation of socialized data offers an unprecedented opportunity for designing customer service measurement systems. In this paper we address the problem of adequately measuring service quality using socialized data. The theoretical basis for the study is the widely used SERVQUAL model. The analysis is based on a database of online reviews generated on the website of the leading price comparison engine in Italy. We use a weakly supervised topic model to extract relevant dimensions of service quality from the user-generated content. Despite its exploratory nature the study offers two contributions. First, it demonstrated that socialized textual data, not just quantitative ratings, provide a wealth of customer service information that can be used to measure the quality offered by service providers. Second, it shows that the distribution of topics in opinions differs significantly between positive and negative reviews. Specifically, we find that concerns about merchant responsiveness dominate negative reviews.