In this extended abstract, we generalize active learning to tasks where a human has to choose which action a to take for a target after observing its covariates and predicted outcomes. An example case is personalized medicine and the decision of which treatment to give to a patient. We show that standard active learning, which is not aware of the final task, would be very inefficient, and we introduce a new problem of decision-making-aware active learning. We formulate the problem as finding the query with the highest information gain for the specific decision-making task, assuming a rational decision-maker. The problem can be solved particularly efficiently assuming an expert able to answer queries about counterfactuals. We demonstrate the effectiveness of the proposed method in a binary outcome decision-making task using simulated data, and in a continuous-valued outcome task on the medical dataset IHDP with synthetic treatment outcomes. The outcomes are predicted using Gaussian processes.