This paper explores the potential of land consolidation for dealing with land fragmentation in Sub-Saharan Africa’s (SSA) rural customary lands – where the intention is to increase food productivity. In SSA’s customary lands, the use of mechanized farming technology and intensive farming techniques have largely failed to increase food productivity. This is despite foreign investment and the interest of the farmers to do so. In many cases, neither the farm parcel structure nor the land tenure arrangements support the use of, or investment in, mechanized equipment. This implies a strong need to deal with the land fragmentation situation. Although land consolidation is argued as an effective response to land fragmentation; its application in SSA’s customary lands has either not been successful, or it has ended up breaking down the customary land tenure arrangements. We argue that past attempts at land consolidation in SSA’s customary lands have failed mainly due to the transfer of European strategies without adequate consideration for the local factors in the planning and implementation, as well as inadequate land information. Land consolidation strategies in Europe have shown that responsible approaches continually considered the changing local factors. There has been a recent push for more responsible approaches to land reform and planning activities that consider social, cultural, and economic factors that were previously not considered. In this paper, the nature and causes of land fragmentation in customary lands will first be explored, then current approaches seeking to increase farm productivity are reviewed. Analysing the problems of land fragmentation in customary lands, the failure to adapt land consolidation approaches in customary lands in the past, and the potential of participatory land administration as an enabling tool, we conclude that responsible approaches are an important component of increasing food productivity in sub-Saharan Africa.