Representation learning is a fascinating aspect of deep learning that is often not examined closely. Convolutional Neural Networks learn representations geared to the target task implicitly during training and those representations are fundamental to the performance of that model. The model’s performance and ability to generalize is directly a function of the properties of those representations and the characteristics of the transformation discovered by the deep networks. In applying deep learning to Seismic Interpretation specifically, we recognise 3 challenges; i) interpretation embodies a number of very different prediction tasks all heavily reliant on how seismic is decomposed and represented; surface picking, fault picking, geobody extraction, and unconformity picking. ii) significant variation between seismic datasets present challenges in training models that generalise properly across them. iii) fundamentally, we want to minimise the need for manual picking and labelling in order to train a machine learning model to carry out seismic interpretation. We propose that learning good quality representations that target different interpretation features (e.g. faults, channels, unconformities) can help us address each of these challenges and that autoencoders are sufficient to learn these representations. In this work, we apply deep convolutional autoencoders to learn representations from seismic that are subsequently used for to interpretation tasks including channel picking and fault detection. We study the learned representations, comparing these against traditional decompositions including fixed frame and data driven methods; namely wavelets, curvelets and Independent Component Analysis and measure their relative performance. Through these comparisons, we develop guidelines for assessing the quality of learned representations that can be used to inform network design and to evaluate training set quality. We also gain insights into how CNN effectively decompose seismic under different constraints. We applied the workflow on datasets from the Norwegian Continental Shelf where we had seismic data calibrated to nearby wells.