Artificial neural networks (ANN) have become a powerful tool for machine learning. Resistive memory devices can be used for the realization of a non-von Neumann computational platform for ANN training in an area-efficient way. For instance, the conductance values of phase-change memory (PCM) devices can be used to represent synaptic weights and can be updated in-situ according to learning rules. However, non-ideal device characteristics pose challenges to reach competitive classification accuracies. In this paper, we investigate the impact of granularity and stochasticity associated with the conductance changes on ANN performance. Using a PCM prototype chip fabricated in the 90nm technology node, we present a detailed experimental characterization of the conductance changes. Simulations are done in order to quantify the effect of the experimentally observed conductance change granularity and stochasticity on classification accuracies in a fully connected ANN trained with backpropagation.