Optimizing hybrid vigor (HV) has been a primary objective in hybrid crop breeding programs. The effective use of DNA markers in this endeavor has been plagued by inconsistent results among researchers who investigated the relationship between genetic similarity (GS) among parents and heterosis in their F1’s. The objectives of this study were to evaluate the relationship between GS among parents and heterosis in F1 for grain yield in japonica rice, and to formulate a strategy for marker-based prediction of heterosis. Forty-five F1 hybrids were generated from a half-diallel design using 10 japonica rice cultivars. Parents and hybrids were grown under medium N and 0-N fertilizers and evaluated for grain yield. The parental varieties were genotyped with 188 SSR markers and/or 129 RAPD primers. Estimates of GS based on all markers were not highly correlated with heterosis. After analyzing the association of each marker to yield heterosis, we were able to identify markers that were significantly associated with HV in this collection of japonica rices. The estimates of GS based on this set of selected markers showed highly significant correlations with HV. In addition, some SSR and RAPD markers were found to be associated with hybrid weakness (HW). This suggests that the heterosis observed in the F1’s could be improved by selectively combining alleles associated with HV and eliminating those associated with HW. A new concept for developing “key markers” is proposed and the feasibility of predicting heterosis through estimation of GS among parents utilizing key DNA markers is discussed.