[This is a post-peer-review, pre-copy edit version of an article in AI & Society. The final authenticated version is available online at: http://dx.doi.org/10.1007/s00146-020-01033-8 or https://rdcu.be/b6avX.] Like previous educational technologies, artificial intelligence in education (AIEd) threatens to disrupt the status quo, with proponents highlighting the potential for efficiency and democratization, and skeptics warning of industrialization and alienation. However, unlike frequently discussed applications of AI in autonomous vehicles, military and cybersecurity concerns, and healthcare, AI’s impacts on education policy and practice have not yet captured the public attention. This paper therefore evaluates the status of AIEd, with special attention to intelligent tutoring systems and anthropomorphized artificial educational agents. I discuss AIEd’s purported capacities, including the abilities to simulate teachers, provide robust student differentiation, and even foster socioemotional engagement. Next, in order to situate developmental pathways for AIEd going forward, I contrast sociotechnical possibilities and risks through two idealized futures. Finally, I consider a recent proposal to use peer review as a gatekeeping strategy to prevent harmful research. This proposal serves as a jumping off point for recommendations to AIEd stakeholders towards improving their engagement with socially responsible research and implementation of AI in educational systems.