AbstractAI systems are quickly being adopted in radiology and, in general, in healthcare. A myriad of systems is being proposed and developed on a daily basis for high-stake decisions that can lead to unwelcome and negative consequences. AI systems trained under the supervised learning paradigm greatly depend on the quality and amount of data used to develop them. Nevertheless, barriers in data collection and sharing limit the data accessibility and potential ethical challenges might arise due to them leading, for instance, to systems that do not offer equity in their decisions and discriminate against certain patient populations or that are vulnerable to appropriation of intellectual property, among others. This paper provides an overview of some of the ethical issues both researchers and end-users might meet during data collection and development of AI systems, as well an introduction to the current state of transparency, interpretability and explainability of the systems in radiology applications. Furthermore, we aim to provide a comprehensive summary of currently open questions and identify key issues during the development and deployment of AI systems in healthcare, with a particular focus on the radiology area.