Using rich linked employer–employee data for (West) Germany between 1996 and 2014, we conduct a decomposition analysis based on recentered influence function (RIF) regressions to analyze the relative contributions of various plant and worker characteristics to the rise in German wage dispersion. Moreover, we separately investigate the sources of between‐plant and within‐plant wage dispersion. We find that industry effects and the collective bargaining regime contribute the most to rising wage inequality. In the case of collective bargaining, both the decline in collective bargaining coverage and the increase in wage dispersion among the group of covered plants have played important roles.