This paper addresses the problem of mitigating the order effects in incremental learning, a phenomenon observed when different ordered sequences of observations lead to different results. A modification of an ILP incremental learning system, with the aim of making it order-independent, is presented. A backtracking strategy on theories is incorporated in its refinement operators, which causes a change of its refinement strategy and reflects the human behavior during the learning process. A modality to restore a previous theory, in order to backtrack on a previous knowledge level, is presented. Experiments validate the approach in terms of computational cost and predictive accuracy.