Modular deep learning has been proposed for the efficient adaption of pre-trained models to new tasks, domains and languages. In particular, combining language adapters with task adapters has shown potential where no supervised data exists for a language. In this paper, we explore the role of language adapters in zero-shot cross-lingual transfer for natural language understanding (NLU) benchmarks. We study the effect of including a target-language adapter in detailed ablation studies with two multilingual models and three multilingual datasets. Our results show that the effect of target-languaResearch goal: How does the inclusion of target-language adapters in multilingual models influence robustness to adversarial examples in zero-shot cross-lingual transfer, as evaluated on XTREME-R's semantic parsing and dependency parsing tasks compared to XTREME-B's question answering and sentence similarity tasks?Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.7/10.