The self-modulation instability is a key effect that makes possible the usage of nowadays proton beams as drivers for plasma wakefield acceleration. Development of the instability in uniform plasmas and in plasmas with a small density up-step is numerically studied with the focus at nonlinear stages of beam evolution. The step parameters providing the strongest established wakefield are found, and the mechanism of stable bunch train formation is identified.