In this paper, we demonstrate new techniques for data representation in the context of deep learning using agglomerative clustering. The results from previous work show that a good number of encoding and decoding filters of layered autoencoders are duplicative thereby enforcing two or more processing filters to extract the same features due to filtering redundancy. We propose a new way to circumvent this problem and our results show that such redundancy is eliminated, yields smaller networks and filters are able to extract distinct features. The concept is illustrated with Sparse Autoenconders (SAE) using MNIST and NORB datasets.