Technical debt (TD) identification tools can find thousands of technical debt items (TDIs) in a software project. Remedying all of them would take months or even years, so prioritization and decision-making are needed to make this process efficient. On the other hand, advances in machine learning over the last few decades have allowed researchers to apply methods to cluster behaviors and identify patterns in software engineering data. In this study, we aim to develop machine learning methods to decide whether and when a given TDI should be paid off in \st{real} software projects. We performed a survey to collect data from Java open-source software projects hosted on GitHub. From the 2,616 survey responses, we created a dataset using three different labeling strategies - "pay or not", 3-classes, and priority. We applied nine well-known machine learning methods over 27 source code metrics to build models to predict if and when a TDI should be paid off. The best methods for determining whether an item should be paid off achieved a mean accuracy of 0.86 and an F1-score of 0.85. For when to make the payment, we applied four approaches. Their performance achieved an accuracy of 0.59 using traditional analysis and 0.83 with tuned analysis for the most flexible method.