The advanced optical inspection of manually placed components on through-hole printed circuit boards demands robust and fast classifiers. To train such classifiers, one needs vast amounts of previously labeled sample images. Datasets like this are currently not available and thus hinder the deployment of deep-learning algorithms in environments like electronics manufacturing. This paper proposes a new architecture, which uses a superposition of active and unsupervised learning to build a problem specific, fully annotated dataset while training a suitable classifier. The system validates human-made annotation by selectively re-asking for a different opinion, to reduce the risk of human error. Our experiments show a simplification of inspection programming in contrast to the existing approaches.