We introduce a novel method for restoring noisy images that requires no noisy model. The proposed method is inspired by a wavelet-based switching smoothness description of Hoelder function spaces. Specifically, we lower the smoothness of the blurred image locally or globally to obtain an enhanced image. Despite the simplicity of our method, significant improvement is reported in the experiment results in terms of image fidelity measure and visual effect.