While Artificial Intelligence (AI) has shown the potential to help governments in making accurate and effective decisions, concerns have been raised on the risk of exacerbating inequality, deepening distrust, and blurring accountability. These have led to research on the implications of governmental AI use in many domains including AI, public administration (PA), and Human-computer interaction (HCI). Research in the fields of AI, PA, and HCI, however, usually take distinct perspectives, mostly in a purely technical or conceptual, less empirical way or lack of specific context way. Therefore there is little systematic and pragmatic attention to identifying the related cross-discipline research areas and synthesize suggestions for empirical study from those areas that can enhance the better practice of AI use in public sector decision-making. This paper studies the use of AI during the process of policymaking, namely AI-Assisted policymaking (AAPM). We first present a multidisciplinary literature review that explores related concepts and theories. We build on this review to propose a conceptual framework that not only considers the complex socio-technical context of AAPM with the unique risks, limitations, and opportunities in the public sector but also the technological capabilities and constraints of AI systems. The framework identifies important variables from various disciplines and their relationships that influence the practice of AAPM. In addition, we advocate visual explanations as a tool for exploration, sensemaking and communication, as well as for ensuring the accountability of AAPM.