One promising approach to peak to average power ratio (PAPR) reduction for orthogonal frequency division multiple access (OFDMA) is the use of reserved tones, which are modulated so as to reduce the PAPR. One problem with this approach is the computationally efficient determination of reserved tone signals that best reduce the PAPR. In this paper we describe a new algorithm for selection of the reserved tone signals. The algorithm results from a differentiable approximation of the PAPR optimisation criterion. We derive a new joint minimisation method capable of reducing multiple peaks simultaneously. It is shown how this can be utilized in a low complexity gradient update algorithm. Simulations show superior performance (lower resultant PAPR) compared to other methods, where system parameters have been motivated by the proposed long term evolution of the 3GPP cellular standard.