Magnetic and electric toroidal dipoles possess interesting properties differing from traditional electric and magnetic dipoles. In order to generate both the magnetic and electric toroidal dipoles simultaneously in one single structure, a permittivity-asymmetric dielectric metasurface is proposed, which is composed of clusters of four high-index dielectric nano-disks with asymmetric permittivity distribution. These two types of toroidal dipole responses can be separately observed at different spectral positions. This study reveals that symmetry-breaking in a broad sense is crucial for exciting toroidal responses, and the proposed metasurface points to a unique routine of exciting and enhancing the toroidal responses, which may be used to realize efficient light–matter interaction in the area of meta-optics.