This study presents a novel approach to analyzing the present value of total profit for fixed and indexed mortgage products in order to determine the optimal mortgage interest rate that would maximize the bank’s expected total profit based on applying the approach used in operations research to the field of finance. The study considers the impact of lending rate, demand, prepayment, and defaults on bank profits and emphasizes the trade-offs between potential gains and losses when setting the lending rate. As such, we not only used a fixed-rate mortgage model or an index mortgage model with the interest rate as the decision variable, but also employed mathematical analysis methods to find out the loan rate that maximizes the present value of the bank’s expected total profit. The findings revealed that an increase in interest rate, loan amount, and demand positively impacted the bank profits, while prepayment had an adverse effect. The study highlights the importance of carefully evaluating various factors that influence revenue in order to arrive at the most appropriate lending rate that will optimize profits. The results provide valuable insights into the optimal mortgage interest rate and the factors that determine the revenue and profits of a bank, with implications for cost–benefit analysis, fixed-rate mortgage, indexed mortgage, lending rate, defaults, and maximum profit. This study contributes to the existing literature on mortgage products. It provides practical implications for banks in managing their mortgage products efficiently in order to enhance their financial performance and recommends optimizing mortgage interest rates for maximum bank profits by taking the lending rate, demand, and prepayment effects into account.