AbstractBackgroundThe increasing prevalence of Artificial Intelligence (AI) language models, exemplified by ChatGPT, has sparked inquiries into their influence on creative writing skills in educational contexts. This study aims to quantitatively investigate whether ChatGPT's use negatively affects university students' creative writing abilities, focusing on originality, content presentation, accuracy, and elaboration in essays. The research adopts an experimental approach to shed light on this concern.ObjectiveThis study aims to quantitatively investigate whether the utilization of ChatGPT, an AI chatbot, adversely affects specific dimensions of creative writing skills among university students, with an emphasis on originality, content presentation, accuracy, and elaboration.MethodThe experimental study involves 600 students from 10 universities, divided into a control and an experimental group (EGp). The EGp incorporates ChatGPT in their creative writing process as an intervention. The study evaluates originality, content presentation, accuracy, and elaboration, utilizing the Wilcoxon Signed‐Rank Test for analysis.Results and ConclusionThe findings reveal a detrimental association between ChatGPT use and university students' creative writing abilities. Analysing both machine‐based and human‐based assessments substantiates earlier qualitative observations regarding ChatGPT's adverse impact on creative writing. This study highlights the necessity of approaching AI integration, particularly in creative writing disciplines, with caution. While AI tools have merits, their integration should be thoughtful, considering the potential drawbacks. These insights inform future research and educational practices, guiding the effective incorporation of AI while nurturing students' writing skills.