We investigate the mechanical response of thin sheets perforated with a square array of mutually orthogonal cuts, which leaves a network of squares connected by small ligaments. Our combined analytical, experimental and numerical results indicate that under uniaxial tension the ligaments buckle out-of-plane, inducing the formation of 3D patterns whose morphology is controlled by the load direction. We also find that by largely stretching the buckled perforated sheets, plastic strains develop in the ligaments. This gives rise to the formation of kirigami sheets comprising periodic distribution of cuts and permanent folds. As such, the proposed buckling-induced pop-up strategy points to a simple route for manufacturing complex morphable structures out of flat perforated sheets.