In this second part of the paper the authors continue studying the regularity properties of the ``minimal'' set \(K\) in the free discontinuity problem ( the pair \((u,K)\) being a quasi minimizer for the functional \(F(u,K)\) - see the review above of the Part I of the paper). Here, the assumption \(| \nabla u| \in L^{2,\lambda}(\Omega)\), admitted in the Part I of the paper is removed owing to a suitable decay lemma. So, the main result of the paper can be stated as follows: any optimal free discontinuity set \(K\) is a \(C^{1,\alpha}\) hypersurface except for a singular set \(S\) satisfying \({\mathcal H}^{n-1}(S) = 0\). A characterization of singular points is also given. It can be exploited to get further information on the dimension and the structure of the set \(S\). The important role in the paper play, on one hand the SBV spaces of special functions of bounded variation introduced by De Giorgi and Ambrosio, and on the other hand two decay estimates. The first one concerns the flatness improvement (showing that the Dirichlet energy controls the mean curvature of \(K\)) and the second decay estimate is concerned with the Dirichlet energy.