The search for better compositions in high entropy alloys is a formidable challenge in materials science. Here, we demonstrate a systematic Bayesian optimization method to enhance the mechanical properties of the paradigmatic five-element Cantor alloy in silico. This method utilizes an automated loop with an online database, a Bayesian optimization algorithm, thermodynamic modeling, and molecular dynamics simulations. Starting from the equiatomic Cantor composition, our approach optimizes the relative fractions of its constituent elements, searching for better compositions while maintaining the thermodynamic phase stability. With 24 steps, we find Fe21Cr20Mn5Co20Ni34 with a yield stress improvement of 58%, and with 72 steps, we find Fe6Cr22Mn5Co32Ni35 where the yield stress has improved by 74%. These optimized compositions correspond to Ni-rich medium entropy alloys with enhanced mechanical properties and superior face-centered-cubic phase stability compared to the traditional equiatomic Cantor alloy. The automatic approach devised here paves the way for designing high entropy alloys with tailored properties, opening avenues for numerous potential applications.