Thanks to the ever-developing educational technologies, more and more educational institutions and testing organizations around the world have been delivering their testing events through computer-based formats. As far as language assessment is concerned, these developments have led to questions on all sorts of validity being raised by educational and assessment researchers, especially regarding the speaking skill. This exploratory paper compares and contrasts the pros and cons of computer-based and face-to-face assessment of speaking from a communicative view of language, by exploring what exactly needs to be assessed in speaking, the effects of the delivery mode and the differences between computerized and human rating. The findings are that face-to-face assessment of speaking is a much more valid format than computer-based assessment from a communicative point of view, but that the latter can provide a partial answer to the bulk administration problem in contexts such as placement or exit tests in educational institutions.