Spiking Neural Networks (SNNs) offer a promising computing paradigm suitable for low-power artificial intelligence. Spike-Timing Dependent Plasticity (STDP) is an unsupervised, biologically-inspired learning rule for SNNs. This work studies the reliability of STDP-trained SNNs under hardware faults which is largely unexplored. We present a thorough fault injection analysis of an STDP-trained SNN designed in Brian 2 simulator for MNIST classification. The analysis introduces faults in neurons and synapses before, during, and after training. We consider both permanent and transient, as well as single and multiple faults. We identify cases where the SNN exhibits inherent fault tolerance, cases where it adapts to faults through training, and cases where fault tolerance mechanisms are required.