In recent years, there has been a shift in software development towards microservice-based architectures. In this paper, we discuss the benefits and challenges of microservices from a performance tester's point of view. Through a series of experiments on the TeaStore application, we demonstrate how microservices affect the performance testing process, and we demonstrate that it is not straightforward to achieve reliable performance testing results for a microservice application. Reproducing the results from this paper consists of two parts, reproducing the performance measurements using TeaStore in the Google Kubernetes Engine and reproducing the analysis conducted in the paper based on the collected measurement data. In order to enable quick reproduction of the experiments, we packaged the scripts for the experiment automation as a docker container. The measurement data collected in this study and the scripts to reproduce the analysis presented in the paper are available in form of a CodeOcean capsule.