This paper x-rayed the perceived issues and challenges associated with teaching and learning of mathematics in Nigerian secondary schools. The paper argued that mathematics is a very important subject in secondary schools and highly needed for manpower and technological development of any nation, and as such, needs proper teaching and learning in order to realize the desirable objectives. Some of the challenges facing the teaching and learning of mathematics in Nigeria include unprofessionally trained teachers, inability of the teachers to understand the students, inadequate instructional materials, lack of mathematics laboratories, the inability of the teachers to use different teaching approaches, chock-a-block classes, inability of the mathematics teachers to make full preparation and proper planning. The paper also tried to look into the poor attitude of the students towards mathematics and school location as factors militating against the teaching and learning of mathematics in Nigeria. The paper, however, concluded with recommendations which can be very useful in finding solutions to the challenges, which include, among others, that competently trained mathematics teachers should be recruited and employed to teach mathematics in secondary schools in Nigeria.