An algorithm is presented for clustering sequential data in which each unit is a collection of vectors. An example of such a type of data is speaker data in a speaker clustering problem. The algorithm first constructs affinity matrices between each pair of units, using a modified version of the Point Distribution algorithm which is initially developed for mining patterns between vector and item data. The subsequent clustering procedure is based on fitting a Gaussian mixture model on multiple random projection matrices. The final class label of each unit is determined by voting from the results of the random projection matrices.