The purpose of this paper is to compare the abilities and deficiencies of various swarm based algorithms for training artificial neural networks. This paper uses seven algorithms, seven regression problems, sixteen classification problems, and four bounded activation functions to compare algorithms in regards to loss, accuracy, hidden unit saturation, and overfitting. It was found that particle swarm optimization is the top algorithm for regression problems based on loss, firefly algorithm was the top algorithm for classification problems when examining accuracy and loss. The ant colony optimization and artificial bee colony algorithms caused the least amount of hidden unit saturation, with the bacterial foraging optimization algorithm producing the least amount of overfitting.