Lower bounds for variances are often needed to derive central limit theorems. In this paper, we establish a lower bound for the variance of Poisson functionals that uses the difference operator of Malliavin calculus. Poisson functionals, i.e. random variables that depend on a Poisson process, are frequently studied in stochastic geometry. We apply our lower variance bound to statistics of spatial random graphs, the $L^p$ surface area of random polytopes and the volume of excursion sets of Poisson shot noise processes. Thereby we do not only bound variances from below but also show positive definiteness of asymptotic covariance matrices and provide associated results on the multivariate normal approximation.