Predicting Question Deletion and Assessing Question Quality in Social Q&A Sites using Weakly Supervised Deep Neural Networks

Predicting Question Deletion and Assessing Question Quality in Social Q&A Sites using Weakly Supervised Deep Neural Networks

Online

Community question answering (CQA) sites, which use the power of collective knowledge, have emerged as popular destinations for complex and personalized questions that require human-human interactions and multiple rounds of clarifications between the asker and the answerer. In this paper, we undertook a threefold task: First, we developed a deep neural network model to automatically predict the questions that are likely to be deleted by the moderators. Second, we hypothesized that there exists a relationship between the question quality and its probability of being deleted by the forum moderators. We developed a deep model using deleted questions and used it for predicting question quality. Our contribution is not limited to developing the predictor model; we also created the gold standard data for question quality assessment. Lastly, we explored the efficiency of different input representations, optimization functions, and neural network models for predicting question quality. When assessing question quality, the results highlight that combining natural language features with word embeddings can result in better performance (higher recall and f-scores) than word embeddings alone. Our model predicted deleted-questions with an accuracy of 97.8% and precision and true positive rates (TPR) above 0.95. While assessing question quality, our model obtained a TPR of 0.841 and a precision of 0.514. This research serves as the first step toward automatic content moderation in CQA sites; identifying poor quality questions would allow askers to improve the quality of questions asked and the moderators to handle a large volume of questions during content moderation.

https://aisel.aisnet.org/hicss-54/dsm/data_mining/5