Inoculation is a widely used method to improve the efficiency of anaerobic digestion (AD) with a high organic load. This study was conducted to prove the potential of dairy manure as an inoculum source for AD of swine manure. Furthermore, an appropriate inoculum-to-substrate (I/S) ratio was determined to improve methane yield and reduce the required time of AD. We carried out 176 days of anaerobic digestion for five different I/S ratios (3, 1, and 0.3 on a volatile solid basis, dairy manure alone, and swine manure alone) of manure, using solid container submerged lab-scale reactors in mesophilic conditions. As a result, solid-state swine manure inoculated with dairy manure could be digested without inhibition caused by ammonia and volatile fatty acid accumulation. The highest methane yield potential was observed in I/S ratios 1 and 0.3, as 133 and 145 mL CH4·g−1-VS, respectively. The lag phase of swine manure alone was more extended, 41 to 47 days, than other treatments containing dairy manure, directly related to tardy startup. These results revealed that dairy manure can be used as an inoculum source for AD of swine manure. The proper I/S ratios leading to successful AD of swine manure were 1 and 0.3.