The dataset consists of over 1.1 million samples of labeled cyclists actions. Every sample consists of two motion history images (MHI), consisting of motion over the past second, from two different cameras, the past trajectory of the cyclist of the last second (50 past positions), and a label of the currently performed action. The samples were extracted from 1,639 video sequences of cyclists moving across an urban intersection at the University of Applied sciences in Aschaffenburg: https://www.th-ab.de/ueber-uns/organisation/labor/kooperative-automatisierte-verkehrssysteme/ausstattung/ The uploaded files consist of a numpy file with MHI, trajectories and labels, and a json file containing 5-fold cross validation/test split. Every MHI has a size 192x192x2. The two channels contain the separate motion contours of the bike and the cyclist. The manually created labels are: straight: cyclists is moving and not turning tr/tl: cyclist is turning left/right move: cyclist is moving with nearly constant velocity and not turning start: cyclist was standing and starts moving starting_movement: first movement of cyclist before starting stop: cyclist was moving/starting and slows down to a halt wait: cyclist is standing hand_signal_left/right: cyclist indicates a turn by hand signal shoulder_check_left/right: cyclist looks over left/right shoulder out_of_saddle: cyclist is standing The numpy file consists of python dictionaries with scenes of the form: {SCENE_NAME: 'mhi_hk1/2': [...], # list of zip compressed, python pickled mhi from cameras 1 and 2 'x/y/z_tracked': [...], # tracked cyclists positions in x/y/z directions, 'x/y/z_smoothed': [...], # smoothed (by rts smoother) cyclists positions in x/y/z directions, 'orientation': [...], # orientation of the cyclists estimated by kalman filters 'ts': [...], # utc timestamps in micro seconds LABEL_NAME: [...], # labels of different actions (0 or 1)} The zipped/pickled mhi can be decompressed using: import numpy as np import pickle import zlib d = np.load('mhi_dataset_0,1,2,3,4,6,9,13,18,26,33,41,49.npy', allow_pickle=True, encoding='latin1').item() for scene_name in d: for i in range(len(d[scene_name]['mhi_hk1']) - 50): # ommit 50 first frames as they have no mhi mhi1 = pickle.loads(zlib.decompress(d[scene_name]['mhi_hk1'][49 + i].encode('latin1')), encoding='latin1') mhi2 = pickle.loads(zlib.decompress(d[scene_name]['mhi_hk2'][49 + i].encode('latin1')), encoding='latin1') This work results from the project DeCoInt 2, supported by the German Research Foundation (DFG) within the priority program SPP 1835: "Kooperativ interagierende Automobile", grant numbers DO 1186/1-2, FU 1005/1-2, and SI 674/11-2. Additionally, the work is supported by "Zentrum Digitalisierung Bayern". Due to privacy laws in germany, we are not permitted to publish raw image data.