Neural networks are essential components of learning-based software systems. However, their high compute, memory, and power re- quirements make using them in low resources domains challenging. For this reason, neural networks are often quantized before deployment. Ex- isting quantization techniques tend to degrade the network accuracy. We propose Counter-Example Guided Neural Network Quantization Re- finement (CEG4N). This technique combines search-based quantization and equivalence verification: the former minimizes the computational re- quirements, while the latter guarantees that the network’s output does not change after quantization. We evaluate CEG4N on a diverse set of benchmarks that include large and small networks. Our technique was successful at quantizing the networks in our evaluation while producing models with up to 72% better accuracy than state-of-the-art techniques.