Recommender systems play a pivotal role in shaping access to information online, yet audits of these systems rarely prioritize the perspectives of those most affected by them: the users. In the light of a growing body of work on recommender system audits, this systematic review examines the extent and forms of user involvement, classifying modes of participation and assessing how they influence audit practices. Our analysis finds that most audits treat users primarily as passive data sources, with only a small fraction engaging directly with their lived experiences. Where direct involvement occurs, it provides insights that purely technical approaches often miss, exposing critical blind spots in current methodologies. We argue that meaningful accountability requires a paradigm shift: sustained and active user participation, enhanced access to data for independent auditors, and stronger interdisciplinary collaboration across research, industry, and civil society. These findings highlight the urgent need for auditing frameworks capable of delivering more transparent and genuinely accountable recommender systems.