Cells in living organisms are dynamic compartments continuously responding to changes in their environment to maintain physiological homeostasis. While basal autophagy exists in cells to aid in the regular turnover of cellular debris, starvation-induced autophagy is a critical cellular response to stress, such as nutritional depletion. However, the deregulation of autophagy is linked to several diseases, such as cancer, and hence constitutes a potential therapeutic target. Image analysis to follow autophagy in cells, especially in high-content screens, has proven to be a bottleneck in the pipeline. Machine learning (ML) algorithms have recently emerged as crucial part of efficiently extracting information from images, thus contributing to a better understanding of the questions at hand. This open dataset contains images of cells under a microscope with cell-specific segmentation masks. Each cell is annotated into either basal or activated autophagy. Furthermore, we applied ML algorithms to process time-lapse high-content live-cell imaging data of cells in different autophagic states.