This work introduces systematic approach for enhancing large language models (LLMs) toaddress Bangla AI mathematical challenges. Through the assessment of diverse LLM configurations, finetuning with specific datasets, and the implementation of Retrieval-Augmented Generation (RAG), weenhanced the model’s reasoning precision in a multilingual setting. Crucial discoveries indicate that customized prompting, dataset augmentation, and iterative reasoning improve the model’s efficiency regardingOlympiad-level mathematical challenges.Keywords: Large Language Models (LLMs), Fine-Tuning, Bangla AI, Mathematical Reasoning, RetrievalAugmented Generation (RAG), Multilingual Setting, Customized Prompting, Dataset Augmentation, Iterative Reasoning, Olympiad-Level Challenges