The ultimate goal of Continuous Integration (CI) is to support developers in integrating changes into production constantly and quickly through automated build process. While CI provides developers with prompt feedback on several quality dimensions after each change, such frequent and quick changes may in turn compromise software quality without Continuous Refactoring (CR). Indeed, recent works emphasized the potential of CI in changing the way developers perceive and apply refactoring. In this paper, we conduct a large-scale empirical study to investigate and gain insight into the potential impacts of CI adoption on developers refactoring practices. Our study involves a corpus of 502,259 commits and 496,820 refactoring instances extracted from 240 open source GitHub projects that adopt Travis CI, a popular online continuous integration framework. Our statistical results show that the refactoring effort has decreased with the use of CI, while we surprisingly found that the refactoring frequency tend to decrease the year following Travis CI adoption. These findings provide further evidence for the presence of several barriers preventing developers from continuously perform refactoring in CI context.