Technical debt refers to incomplete or temporary workarounds that allow us to speed software development in the short term at the cost of paying a higher price later on. Recently, studies have shown that technical debt can be detected from source code comments, referred to as self-admitted technical debt. Researchers have examined the detection, classification and removal of self-admitted technical debt. However, to date there is no empirical evidence on the impact of self-admitted technical debt on software quality. Therefore, in this paper, we examine the relation between self-admitted technical debt and software quality by investigating whether (i) files with self-admitted technical debt have more defects compared to files without self-admitted technical debt, (ii) whether self-admitted technical debt changes introduce future defects, and (iii) whether self-admitted technical debt-related changes tend to be more difficult. We measured the difficulty of a change using well-known measures proposed in prior work such as the amount of churn, the number of files, the number of modified modules in a change, as well as the entropy of a change. An empirical study using five open source projects, namely Hadoop, Chromium, Cassandra, Spark and Tomcat, showed that: i) there is no clear trend when it comes to defects and self-admitted technical debt, although the defectiveness of the technical debt files increases after the introduction of technical debt, ii) self-admitted technical debt changes induce less future defects than none technical debt changes, however, iii) self-admitted technical debt changes are more difficult to perform, i.e., they are more complex. Our study indicates that although technical debt may have negative effects, its impact is not only related to defects, rather making the system more difficult to change in the future.