The paper introduces a hybrid evolving architecture for dealing with incremental learning. It consists of two components: resource allocating neural network (RAN) and growing Gaussian mixture model (GGMM). The architecture is motivated by incrementality on one hand and on the other hand by the possibility to handle unlabeled data along with the labeled one, given that the architecture is dedicated to classification problems. The empirical evaluation shows the efficiency of the proposed hybrid learning architecture.