Thermal design and optimization for reverse water gas shift (RWGS) reactors is particularly important to fuel synthesis in naval or commercial scenarios. The RWGS reactor with irreversibilities of heat transfer, chemical reaction and viscous flow is studied based on finite time thermodynamics or entropy generation minimization theory in this paper. The total entropy generation rate (EGR) in the RWGS reactor with different boundary conditions is minimized subject to specific feed compositions and chemical conversion using optimal control theory, and the optimal configurations obtained are compared with three reference reactors with linear, constant reservoir temperature and constant heat flux operations, which are commonly used in engineering. The results show that a drastic EGR reduction of up to 23% can be achieved by optimizing the reservoir temperature profile, the inlet temperature of feed gas and the reactor length simultaneously, compared to that of the reference reactor with the linear reservoir temperature. These optimization efforts are mainly achieved by reducing the irreversibility of heat transfer. Optimal paths have subsections of relatively constant thermal force, chemical force and local EGR. A conceptual optimal design of sandwich structure for the compact modular reactor is proposed, without elaborate control tools or excessive interstage equipment. The results can provide guidelines for designing industrial RWGS reactors in naval or commercial scenarios.