The authors have published earlier a novel technique for the supervised training of feed-forward artificial neural networks using the Harmony Search algorithm. This paper proposes a parallel and distributed implementation method to speedup the execution time to address the training of larger pattern-classification benchmarking problems. The proposed method is a hybrid technique that adopts form the merits of two common parallel and distributed training methods, namely network partitioning and pattern partitioning. Experimentation is carried out on a large pattern-classification benchmarking problem using two Master-Slave parallel systems, a homogeneous system using a cluster computer and a heterogeneous system using a set of commodity computers connected via switched network. Results show that the proposed method attains a considerable speedup in comparison to the sequential implementation.