Abstract Background Cardiovascular diseases (CVDs) are the leading cause of morbidity and premature mortality globally. While the relationship between indicators of physical fitness and arterial structure and stiffness are reasonably well-studied in adults, these associations in children and adolescents remain less understood. The aim of this study was to investigate longitudinal associations of cardiorespiratory fitness, muscular fitness and motor fitness with arterial structure and stiffness from childhood to adolescence. Results Higher mean value of VO 2peak /LM from childhood to adolescence was associated with higher carotid intima-media thickness (cIMT) at 8-year follow-up (β = 0.184, 95% confidence interval [CI] = 0.019 to 0.350). Better performance in sit-up test at baseline was associated with lower cardio-ankle vascular index (CAVI) (β = − 0.219, 95% CI = − 0.387 to − 0.051) at 8-year follow-up, and higher mean sit-up performance from baseline to 8-year follow-up was associated with lower carotid-femoral pulse-wave velocity (cfPWV) (β = − 0.178, 95% CI = − 0.353 to − 0.003) and CAVI (β = − 0.190, 95% CI = − 0.365 to − 0.016) at 8-year follow-up. Also cross-sectionally, better sit-up performance at 8-year follow-up was associated with lower cfPWV (β = − 0.232, 95% CI = − 0.411 to − 0.054) and CAVI (β = − 0.185, 95% CI = − 0.365 to − 0.005) and higher carotid artery distensibility (β = 0.165, 95% CI = 0.004 to 0.327) at 8-year follow-up. Most of the associations were explained by body fat percentage (BF%). Conclusions Physical fitness had a weak if any association with indicators of arterial structure and arterial stiffness in adolescence. BF% largely explained the associations of higher VO 2peak /LM with higher cIMT and better sit-up performance with lower arterial stiffness in adolescents. Therefore, preventing adiposity rather than improving CRF should be addressed in public health strategies to prevent CVDs in general paediatric populations. Key Points Better sit-up performance was associated with lower arterial stiffness, but the association was largely explained by body fat percentage. Lower body muscular strength, handgrip strength, or motor fitness was not associated with arterial stiffness or carotid artery intima-media thickness. Measures other than cardiorespiratory fitness, muscular fitness, or motor fitness, such as adiposity, should be used to screen children and adolescents at increased risk of cardiovascular diseases.