This paper analyzes ethical concerns found in users' app store reviews. We performed this study because ethical concerns in mobile application (app) software are widespread, pose severe threats to end-users and society, and lack systematic analysis and methods for detection and classification. In addition, app store reviews allow practitioners to collect users' perspectives, crucial for identifying software flaws, from a geographically distributed and large-scale audience. For our analysis, we collected five million user reviews, developed a set of ethical concerns representative of user preferences, and labeled a sample of these reviews. We found that users highly report concerns regarding inappropriate content, accountability, and discrimination, user reviews with ethical concerns are longer, more popular, and lowly rated, and there is high automation potential for this task. Therefore, these results point to app store reviews being a fruitful channel for practitioners to utilize during software evolution.