Lots of organizations use subscription business models. However, with increasing competition and technological progress switching costs for customers are decreasing. This development can translate to serious issues for subscription-based businesses, requiring action. Traditionally, businesses used mailings or calls, which are costly, time-consuming and often not effective. In this research-in-progress paper, we explore conversational churn prevention as a potential remedy. We present a conversational agent with persuasive design features (e.g., nudges) and first results from a pre-study. We conduct an in-between subject experiment and interviews for our mixed-methods evaluation of our pre-study. Our work contributes to theory, by presenting more insights into the interaction quality of conversational agents in the context of churn prevention of digital services and the role of persuasive design. We support practitioners, by guiding them towards more effective use of conversational agents to improve their services and to predict churn.