The fast and increasingly natural adoption of handheld devices such as smartphones and tablets into everyday life is a rapidly growing social phenomenon which is also believed to have vast potential for foreign language learning. The small-scale study presented in this article contributes to the discussion of whether handheld devices should be further integrated into or eliminated from the classroom by presenting part of the results of a small-scale study conducted in a German secondary English as a foreign language classroom. The study examined the impact of a mobile vocabulary learning app on secondary school students’ vocabulary performance, also giving some indications about the students’ development of foreign language learner autonomy. Although the results of the study showed an unexpectedly limited use of the app, they allow assumptions to be made about the reasons which prevented students from using the app more frequently which can influence future researchers’ and practitioners’ app choices. The first section of this article briefly outlines the theoretical framework of the study, particularly focussing on the potential of computer-assisted and mobile-assisted language learning for enhancing students’ foreign language skills and learner autonomy. The ensuing section then provides some background on investigating students’ foreign language vocabulary learning. This is followed by a presentation of the research design, including the main features of the app (Socrative) used in this study. The next section then critically discusses some of the major research results. The contribution concludes by summarising some of the outcomes of the study and outlining some gaps for further research.