Tugboats are indispensable for ensuring the safe and efficient berthing and unberthing of large vessels, and their scheduling policies have a direct impact on port efficiency and operating costs. To overcome the limitations of conventional single-objective optimization approaches, this paper develops a multi-objective, mixed-integer linear programming (MILP) model that establishes a symmetric consideration by simultaneously minimizing total operating cost and operation time. In addition, a hybrid optimization framework that employs a Jaya algorithm integrated with Q-learning (Jaya-QL) is introduced. Its Q-learning-driven adaptive mechanism achieves a symmetric balance between global exploration and local exploitation, mitigating premature convergence in the Jaya algorithm. Experimental results show that Jaya-QL achieves average reductions of 17.5% in total cost and 0.65% in total time compared with the Artificial Bee Colony (ABC), Quantum Particle Swarm Optimization (QPSO), Ant Colony Optimization (ACO), Genetic algorithm (GA) and Jaya algorithms. Moreover, it demonstrates superior convergence accuracy and solution diversity, offering a practical and effective decision support tool for tugboat scheduling in modern port operations.