This manuscript presents an approach to the automatic generation of open cloze exercises based on arbitrary English text. The exercise format is similar to the open cloze test used in Cambridge English certificate exams (FCE, CAE, CPE). The presented method also makes it possible to adjust the difficulty of the resulting exercises to better suit specific proficiency levels. Three experiments were conducted to evaluate the usefulness of the machine-generated exercises, compare them with authentic Cambridge English tests and study the difficulty-setting capabilities. The experiments showed that the generation method used was quite effective. With some customization, the method can be applied to generating similar exercises for other languages.