Stochastic optimization problems depending on random elements through the corresponding probability measure appear in stochastic programming practice very often. Evidently, the probability measure can be considered as a parameter of such a problem. Consequently, a stability analysis with respect to a probability measure perturbation should yield useful results. The paper is devided into two parts. First, the stability of recourse problems is studied. The Wasserstein matrix is employed to obtain the Lipschitz property of the optimal value and the upper semicontinuity of the optimal solutions. The cases of linear and quadratic recourse are studied in particular. The second part of the paper is devoted to stochastic programming problems with (several joint) probabilistic constraints. There a suitable discrepancy is chosen to obtain (local) upper semicontinuity of the optimal solutions set and (local) Lipschitz continuity of the optimal value too. Special attention is paid to the class of \(r\)-convex probability measures in this part. The achieved results are applied to empirical measures in both considered cases. Let us finally point out that the presented results extend some previous ones on this topic by the same authors.