Crowd behaviour analysis is a challenging task in computer vision, mainly due to the high complexity of the interactions between groups and individuals. This task is particularly crucial given the magnitude of manual monitoring required for effective crowd management. Within this context, a key challenge is to conceive a highly generic, fine and context-independent characterisation of crowd behaviours. Since current datasets answer only partially to this problem, a new dataset is generated, with a total of 11 crowd motion patterns and over 6000 video clips with an average length of 100 frames per sequence. We establish the first baseline of crowd characterisation with an extensive evaluation on shallow and deep methods. This characterisation is expected to be useful in multiple crowd analysis circumstances, we present a new deep architecture for crowd characterisation and demonstrate its application in the context of anomaly classification.