The audit market comprises a small group of large firms and a big group of smaller firms. We build upon the classic model presented by Magee and Tseng (1990) to shed light on auditor market structure. In particular, we focus on the perceived reputation cost borne by auditors that agree with managers on controversial accounting issues. We claim that when perceived reputation cost depends on the weight of a specific client relative to total income, larger audit firms can charge higher fees for the same audits, and clients are willing to pay those higher fees due to the premium paid by investors when audits are perceived to be of higher quality. Moreover, we find that market concentration produces a suboptimal output in which client firms cannot choose the auditor that maximizes their profits, and auditors cannot extract the maximum potential surplus, producing a social loss in the audit market.