I present a study of the mass-metalliciy (MZ) relation in a redshift range of z $\sim 0.3-3.4$ using 33 star-forming galaxies selected by Gamma Ray bursts (GRBs) with emission metallicity measurements. GRBs are beacons of star-forming galaxies up to very high redshifts. The detectability of these extremely bright anddust-penetrating explosions is independent of the brightness and dust content of their host galaxies. Hence faint galaxies are not missed when selected by GRBs. I show that GRB host galaxies remarkably follow the MZ relation of the general star-forming population with considering the redshift evolution of the MZ relation. While our results confirm the MZ relation and its evolution at $z>2$, they also suggest GRB hosts as obvious candidates for studying the MZ relation and its evolution at high redshifts. The presence of GRB afterglows provide accurate metallicity measurements for GRB host galaxies up to very high redshifts (z$\sim$6). These measurements are based on absorption profiles and hence do not suffer from calibration issues. In addition it is confirmed now that though at $\rm z<1.5$ GRB hosts appear to follow the low mass end of the luminosity function, at higher redshifts they sample the star-forming galaxy population. I discuss the possibility of using GRB hosts with absorption metallicity measurements to investigate the MZ relation at high redshifts.