The paper presents a possible enhancement of entropy-based classifiers to handle problems, caused by the class imbalance in the original dataset. The proposed method was tested on synthetic data in order to analyse its robustness in the controlled environment with different class proportions. As also the proposed method was tested on the real medical data with imbalanced classes and compared to the original classification algorithm results. The medical field was chosen for testing due to frequent situations with uneven class ratios.