The proliferation of tools that enable anyone to create visualizations of their data, even with limited experience or skills, has made data visualization more accessible than ever before. This is true in its use in both teaching and learning, as data visualization has increasingly taken on an important pedagogical role in the classroom and in scholarly research. However, with this proliferation of tools there has been a concomitant awareness that visualization needs to be employed through a critical lens that acknowledges its constructedness as explanatory medium and as a product of situated knowledges. Here, I describe one approach to teaching this notion of constructedness via a framework oriented around information literacy, which encourages critical engagement with data, the tools we use to interrogate them, and the visualizations we design to represent them. I describe this approach through a collection of "critical dichotomies" used to evaluate the authority and value of visualizations, which are mapped to a subset of the core information literacy competencies defined in the ACRL Framework for Information Literacy in Higher Education. To put these dichotomies into practice, I further describe an interactive activity called "Choose Your Own Adventure, with Data Visualization," in which participants are given paper and markers to create booklets in the style of Choose Your Own Adventure books and asked to consider the relationship between active choices in the design process of a visualization and how a given visualization is interpreted. In the process, I explore how this framework can encourage us all, as critical practitioners of visualization, to think about the practical relationship between data visualization and information literacy more generally.