This Viewpoint explores challenges within the neurodiversity framework, with a particular focus on autism, and discusses three critical aspects: the risk of epistemic injustice, the balance between over and undermedicalization, and the terminological complexities associated with the "neuro-" prefix. It underscores the importance of a balanced approach that avoids overmedicalization while providing essential support, addresses concerns about the indiscriminate use of "neurodiverse", questions the terminology on neurodiversity, and suggests considering a broader term like "biopsychosocial diversity". Ultimately, this Viewpoint advocates for a measured approach to neurodiversity, emphasizing historical context and a diverse perspective to foster collaboration between cognitive science and behavior research.