This paper describes the use of computer vision to support the operation of a handheld projector, and describes four applications. Projectors in the past have been used as fixed devices, but the latest generation of ‘pocket projectors’ is small and portable. We demonstrate the feasibility of using a projector held in the hand, and the types of applications that can be done with a handheld projector. We attach a camera to the projector to support its operation in two ways. Firstly, vision is used to recover the motion of the projector relative to the display surface. A handheld projector with motion recovery allows a range of interesting functionality, and we show how web-browsing can be done with a handheld projector, complete with mouse interaction and text-entry. Secondly, we use the camera to process information about the projection surface – for example we demonstrate an application that allows a user to attach digital information to a physical texture, and later to recover and view the digital data via recognition of the texture.