In order to help address problems of information overload in digital imagery task domains, we have developed an interactive approach to the capture and reuse of image context information. Our framework models different aspects of the relationship between images and the domain tasks that they support by monitoring the interactive manipulation and annotation of task-relevant imagery. In particular, a strong focus on task context serves to ground image annotations in domain specific goals. This contrasts with prevalent annotation schemes that focus on what individual images contain but that provide no context for which, if any, of those aspects are important to users. Our work attempts to leverage a measure of the user's intentions with regard to tasks that they address. We analyze human-computer interaction information that enables us to infer why image contents are important in a particular context and how specific images have been used to address particular domain goals.