While vision in living beings is an active process where image acquisition and classification are intertwined to gradually refine perception, much of today’s computer vision is build on the inferior process of passive image acquisition and subsequent classification, which is not only inefficient but also lacks the ability to adapt to changing environments. This paper proposes a novel approach that combines reinforcement learning with supervised deep learning to enable active vision, where the agent learns to acquire and classify images in a way that maximizes its reward. The proposed approach is evaluated on a variety of tasks, including object detection, segmentation, and tracking, and shows significant improvements over traditional methods.