Modular multiplication is the key ingredient needed to realize most public-key cryptographic primitives. In a modular setting, multiplications are carried in two steps: namely a usual integer arithmetic followed by a reduction step. Progress in any of these steps naturally improves the modular multiplication but it is not possible to interleave the best algorithms of these stages. In this study, we propose architectures for recently proposed method of interleaving the Karatsuba-Ofman multiplier and bipartite modular reduction on the upper most layer of Karatsuba-Ofman's recursion. We manage to come up with a high performance modular multiplication architecture by taking the advantage of a fast multiplication and a parallel reduction method.