The importance of English as a lingua franca requires students to master English. However, many students still cannot speak English, especially those who live in a country where English is a foreign language. For example, Indonesian students still face numerous problems in learning English, especially speaking. Hence, this study aimed to determine the factors that influence students' speaking performance from teachers' and students' perspectives in learning English speaking to assist students in overcoming difficulties in learning to speak English. Utilizing a survey, a total of one hundred eighty-nine students (n=189) and six teachers (n=6) in SMAN 1 Sape participated in this study. The main research instrument used was a questionnaire. The results showed that the teachers' and students' perspectives on speaking performance were quite similar. Regarding the problem of speaking performance, teachers and students agreed that (1) students were worried that making mistakes was the biggest problem and (2) they were shy to speak English. Moreover, in dealing with factors that influence speaking performance, teachers and students agreed that the highest factor was 1) lack of confidence as they were not familiar with English, (2) lack of time for preparation, and (3) lack of motivation to speak.