Dielectric properties of two series of magnetoactive elastomers (MAEs) based on a soft silicone matrix containing 35 vol% of magnetic particles were studied experimentally in a wide temperature range. In the first series, a hybrid filler representing a mixture of magnetically hard NdFeB particles of irregular shape and an average size of 50 μm and magnetically soft carbonyl iron (CI) of 4.5 μm in diameter was used for MAE fabrication. MAEs of the second series contained only NdFeB particles. The presence of magnetically hard NdFeB filler made it possible to passively control MAE dielectric response by magnetizing the samples. It was shown that although the hopping mechanism of MAEs conductivity did not change upon magnetization, a significant component of DC conductivity appeared in the magnetized MAEs presumably due to denser clustering of interacting particles resulting in decreasing interparticle distances. The transition from a non-conducting to a conducting state was more pronounced for hybrid MAEs containing both NdFeB and Fe particles with a tenfold size mismatch. Hybrid MAEs also demonstrated a considerable increase in the real part of the complex relative permittivity upon magnetization and its asymmetric behavior in external magnetic fields of various directions. The effects of magnetic filler composition and magnetization field on the dielectric properties of MAEs are important for practical applications of MAEs as elements with a tunable dielectric response.