We theoretically investigate the enhancement of thermoelectric cooling performance in thermoelectric refrigerators made of materials with inhomogeneous thermal conductivity, beyond the usual practice of enhancing thermoelectric figure of merit (ZT) of materials. The dissipation of the Joule heat in such thermoelectric refrigerators is asymmetric which can give rise to better thermoelectric cooling performance. Although the thermoelectric figure of merit and the coefficient-of-performance are slightly enhanced, both the maximum cooling power and the maximum cooling temperature difference can be enhanced significantly. This finding can be used to increase the heat absorption at the cold end. We further find that the asymmetric dissipation of Joule heat leads to thermal rectification.