Let n be a fixed natural number. Menger algebras of rank n, which was introduced by Menger, K., can be regarded as the suitable generalization of arbitrary semigroups. Based on this knowledge, an interesting question arises: what a generalization of ternary semigroups is. In this article, we first introduce the notion of ternary Menger algebras of rank n, which is a canonical generalization of arbitrary ternary semigroups, and discuss their related properties. In the second part, we establish the so-called a diagonal ternary semigroup which its operation is induced by the operation on ternary Menger algebras of rank n and then investigate their interesting properties. Moreover, we introduce the concept of homomorphism and congruences on ternary Menger algebras of rank n. These lead us to study the quotient ternary Menger algebras of rank n and to investigate the homomorphism theorem for ternary Menger algebra of rank n with respect to congruences. Furthermore, the characterization of reduction of ternary Menger algebra into Menger algebra is presented.