The aim of the Auto-PCOS classification challenge is to provide an opportunity for the development, testing and evaluation of Artificial Intelligence (AI) models for automatic PCOS classification of healthy and un-healthy frames extracted from ultrasound videos. This challenge encompasses diverse training and test datasets, fostering the creation of vendor-agnostic, interpretable, and broadly applicable AI models. The PCOSGen dataset is first of its kind, consists of different training and test datasets which have been collected from multiple internet resources like YouTube, ultrasoundcases.info, and Kaggle. PCOSGen-train consists of 3200 healthy and un-healthy instances. The training dataset have been medically annotated with the help of experienced gynaecologist based in New Delhi, India.The testing dataset will be released on 15 January 2024.