This is the dataset released with the paper titled: "Understanding and Detecting Hateful Content using Contrastive Learning". We release our dataset in four CSV files. These files contain the textual and visual dataset and the textual and visual ground truth we obtained after our manual annotations. For a detailed description of every column in the CSV structure, along with the type of the value, please read the readme.pdf file provided with this dataset. The images are stored in the zip files. If you find our dataset useful, please cite our paper: @inproceedings{gonzalez2023understanding, title={Understanding and Detecting Hateful Content using Contrastive Learning}, author={Gonz{\'a}lez-Pizarro, Felipe and Zannettou, Savvas}, booktitle={17th International AAAI Conference On Web And Social Media (ICWSM), 2023}, year={2023} } In case of questions, please do not hesitate to contact us: felipegp[at]cs.ubc.ca (https://gonzalezf.github.io)