Recent hype of ChatGPT following user interactions reinforces the perception of Chatbots as new apps. In this research, we adopt the software engineering perspective from the perspective of developer communities to study how AI advances have influenced the chatbot development process, thereby creating value for businesses. We obtain posts from StackOverflow using several keywords related to chatbots by employing StackExchange API. To measure and study the development complexity of IT artefacts, we plan to develop and employ a novel machine learning-based approach using StackOverflow posts and metadata. Further, we plan to analyze the development complexity of chatbots for a period covering the timelines of each category of chatbots: rule-based, retrieval-based AI, and generative AI chatbots. This analysis can provide meaningful insights into the role of AI in chatbot artefacts. This research-in-progress study can significantly contribute to IS research and offer practical implications for building and hosting platforms and other stakeholders.