This dataset consists of color and depth videos of Panda robot motions and their corresponding joint and Cartesian trajectories. The dataset also includes the trajectories of a receiver robot for the purpose of an object handover. Each motion sample comprises 6 files (RGB video, depth video and 4 giver/receiver trajectories in time series form). Total number of motion samples: 38393. Structure: MPEG-4 videos of robot motion and corresponding Python serialized (or ���pickled���) files, containing joint and Cartesian trajectories. Dataset is divided into four parts: simulation dataset (PandaHandover_Sim.zip), real train dataset (PandaHandover_Real_Train.zip), real validation dataset (PandaHandover_Real_Val.zip), real test dataset (PandaHandover_Real_Test.zip). Extract using 7-Zip or similar software. Video files (.avi) can be opened using VLC media player or any other video player that supports MPEG-4 codec. The .pkl files can be loaded using Python (>=3.7) and the Python library Pandas (>=1.1.3).