Star-forming galaxies at high redshift (z=1-3) have irregular, clumpy morphologies. These are best explained by a violent, ubiquitous gravitational instability in gas-rich disks, causing fragmentation in giant clumps and other irregular features. The instability can rapidly redistribute the baryons within galaxies, by inward migration of the giant clumps and other rapid inflows of mass. The promotes the growth of central bulges and other galactic components, and models predict that the resulting properties may finely match those observed in today’s Milky Way-like spirals. The instability-driven inflow may also feed the central black hole in a moderate but steady mode and drive a high fraction of active galaxies, promoting earlier supermassive black hole growth. At the same time, intense stellar feedback in the giant gas-rich clumps and from the active nucleus drive intense gas outflows that regulate the galaxy mass to realistic levels, but the outflows are largely decoupled from the dense gas phases, which continue to form stars continuously. Hence the high-redshift progenitors of massive galaxies evolve in a steady unstable state lasting a couple of Gyr during which they build their fondamental structural components before evolving secularly into modern, quasi-stable spiral disks and spheroids.