CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360° Videos ## General Information We develop the CEAP-360VR dataset to address the lack of continuously annotated behavioral and physiological datasets for 360 video VR affective computing. Accordingly, this dataset contains a) questionnaires (SSQ, IPQ, NASA-TLX); b) continuous valence-arousal annotations; c) head and eye movements as well as left and right eye pupil diameters while watching videos; d) peripheral physiological responses (ACC, EDA, SKT, BVP, HR, IBI). Our dataset also concludes the data pre-processing, data validating scripts, along with dataset description and key steps in the stage of data acquisition and pre-processing. ## Dataset Structure The CEAP-360VR folder contains the following six subfolders 1_Stimuli 2_QuestionnaireData 3_AnnotationData 4_BehaviorData 5_PhysioData 6_Scripts The following is a detailed description of each sub-file: 1_Stimuli - VideoThumbNails contains the eight thumbNails for each video (.jpg) - VideoInfo.json contains the detailed information for eight videos 2_QuestionnaireData - PXX_Questionnaire_Data.json (X = 1, 2, ..., 32) contains questionnaire data for each participant 3_AnnotationData - Raw contains the raw annotation data captured from the Joy-Con joystick for each participant - Transformed contains the transformed valence-arousal data generated from the raw data for each participant - Frame contains the re-sampled annotation data from the transformed data for each participant 4_BehaviorData - Raw contains the raw behavior data captured from the HTC VIVE Pro Eye Tobii Device for each participant - Transformed contains the transformed heam/eye movement data (pitch/yaw) generated from the raw data, as well as pupil diameter data for each participant - Frame contains the re-sampled behavior data generated from the transformed data for each participant - HM_ScanPath contains the head scanpath data generated from the transformed data for each participant - EM_Fixation contains the eye gaze fixation data generated from the transformed data for each participant 5_PhysioData - Raw contains the raw physiological data captured from the Empatica E4 wristband for each participant - Transformed contains the transformed physiological data generated from the raw data for each participant - Frame contains the re-sampled physiological data from the transformed data for each participant 6_Scripts - Unity Project contains the complete project of our user-controlled experiment (Unity 2018.4.1f1, HTC VIVE Pro Eye HMD) - Data Processed contains scripts that undertake the pre-processing steps for converting the raw data to the transformed/frame data in the transformed and frame folders. conatins scripts for continuous annotation, behavior and physiological data analysis and visualization. - CEAP-360VR_Baseline contains scripts to generate processed behavioral and physiological data with V-A labels for deep learning experiments and features for machine learning experiments. contains scripts to run ML and DL experiments under both subject-dependent and subject-independent model. ## Dataset Description The CEAP-360VR Dataset [Description.pdf](https://github.com/cwi-dis/CEAP-360VR-Dataset/blob/master/CEAP-Dataset%20Description.pdf) introduces the dataset description and key steps in the stage of data acquisition and pre-processing. ## Dataset License CEAP-360VR dataset is licensed under a [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license]. ## Citation Please cite our paper in any published work that uses this dataset as follows: - Plain Text T. Xue, A. El Ali, T. Zhang, G. Ding, and P. Cesar, "CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360° Videos," in IEEE Transactions on Multimedia, doi: 10.1109/TMM.2021.3124080. - BibTex @ARTICLE{Xue2021CEAP-360VR, author={Xue, Tong and Ali, Abdallah El and Zhang, Tianyi and Ding, Gangyi and Cesar, Pablo}, journal={IEEE Transactions on Multimedia}, title={CEAP-360VR: A Continuous Physiological and Behavioral Emotion Annotation Dataset for 360° Videos}, year={2021}, volume={}, number={}, pages={1-1}, doi={https://doi.org/10.1109/TMM.2021.3124080}} ## Usage 1. We have performed the time alignment of different types of data and videos for each participant, as well as the proceesing scripts that can be used to generate both the transformed and frame data. Researchers can run their analysis methods on them. 2. For researchers who want to try other data processing methods, you can directly use the raw data. ## About The CEAP-360VR Dataset is maintained by Key Laboratory of Digital Performance and Simulation Technology at Beijing Institute of Technology and the Distributed & Interactive Systems (DIS) research group at Centrum Wiskunde & Informatica . Contact the authors - Tong Xue: xuetong@bit.edu.cn, xue.tong@cwi.nl - Abdallah El Ali: abdallah.el.ali@cwi.nl