The different growth modes and strain relaxation processes in strained layer heteroepitaxy are studied for the case of molecular beam epitaxy of 2.1% lattice-mismatched EuTe on PbTe (111). Due to strain-induced coherent islanding, at high growth temperatures the growth proceeds in a Stranski-Krastanow growth mode, leading to the formation of regularly spaced three dimensional islands, which can relaxe a significant part of the misfit strain by elastic deformation. Due to the existence of a critical surface corrugation length, strain induced 3D islanding can be kinetically suppressed, and thus, at low growth temperatures a 2D layer-by-layer epitaxial growth mode can be sustained. In such a case, strain relaxation can occur only by the formation of misfit dislocations at the layer-substrate interface. Using ultra-high vacuum scanning tunneling microscopy, it is shown that the injection of misfit dislocations at the critical layer thickness causes the appearance of monolayer surface step lines along the dislocation glide line directions. During the course of strain relaxation, the pile-up of misfit dislocations leads to the formation of multiple step surface elevation leading to a considerable roughening of the epitaxial surface. The course of this surface roughening can be understood by taking into account the kinetics of misfit dislocation formation.