We propose a distributed differential space-time coding scheme for two-way relaying over unknown block Rayleigh fading channels. The accent is on a two-time slots scheme, where in the first stage both terminals transmit their signals to the relays and in the second stage the relays broadcast to the terminals. Although we do not assume channel knowledge neither at the relays nor at the terminals, we show that it is possible for the terminals to subtract their contribution from the broadcasted signal and decide on the desired signal from the other terminal. The performance analysis reveals that the two-time slots differential scheme offers advantage over the traditional four-time slots differential scheme.