Adaptive boosting (Adaboost) is a typical ensemble learning algorithm, which has been studied and widely used in classification tasks. Traditional Adaboost algorithms ignore the sample weights while selecting the most useful features, and most of them ignore the fact that the performances of weak classifiers on each category are always different. On this basis, a weighted feature selection and category classification confidence based Adaboost algorithm is proposed in this paper. The first contribution, is that we propose a weighted feature selection to select the most useful features, which can both distinguish the majority of all samples and the previous misclassified samples. The second contribution, is that we improve the traditional error rate calculation method and propose a category based error rate calculation method to combine the classification abilities of Adaboost on different categories. A detailed performances comparison of various Adaboost algorithms are carried out on eight typical datasets. The experimental results show that the proposed algorithm obtains significant improvement on classification accuracy compared to typical Adaboost algorithms when different datasets especially the unbalanced datasets are used.