We have seen in the past years the flourishing of machine and deep learning algorithms in several applications such as image classification and segmentation, object detection and recognition, among many others. This was only possible, in part, because datasets like ImageNet - with +14 million labeled images - were created and made publicly available, providing researches with a common ground to compare their advances and extend the state-of-the-art. Although we have seen an increasing interest in machine learning in geosciences as well, we will only be able to achieve a significant impact in our community if we collaborate to build such a common basis. This is even more difficult when it comes to the Oil & Gas industry, in which confidentiality and commercial interests often hinder the sharing of datasets to others. In this letter, we present the Penobscot interpretation dataset, our contribution to the development of machine learning in geosciences, more specifically in seismic interpretation. The Penobscot 3D seismic dataset was acquired in the Scotian shelf, offshore Nova Scotia, Canada. The data is publicly available and comprises pre- and pos-stack data, 5 horizons and well logs of 2 wells. However, for the dataset to be of practical use for our tasks, we had to reinterpret the seismic, generating 7 horizons separating different seismic facies intervals. The interpreted horizons were used to generated +100,000 labeled images for inlines and crosslines. To demonstrate the utility of our dataset, results of two experiments are presented. Dataset contents Crosslines: Classes: 7 Slices used for training: 289 Records per class for training: 16706 Slices used for testing: 192 Records per class for testing: 1000 Inlines: Classes: 7 Slices used for training: 358 Records per class for training: 14988 Slices used for testing: 238 Records per class for testing: 1000