Poverty in Africa is primarily rural concentrated, about 75% of the poor population live in rural areas and draws their livelihood and food from agriculture. The Sub-Saharan African region is home to more than quarter of a billion people living in extreme poverty, with the Eastern and Southern Africa having the world’s highest concentrations of poor people. The renewed focus on the poverty reducing potential of agricultural productivity accentuate from the fact that the incidence of poverty in Sub-Saharan Africa is increasing faster than the population. The study examined the effect of agricultural productivity on poverty reduction in Africa using the dynamic panel data approach estimated using the System-GMM technique for the period 1991-2015. The conceptual framework of the study identified three main linkages via which agricultural productivity translates to poverty reduction; this include: i. income empowerment, ii. Market expansion, and iii. Sustenance enhancement. The empirical result suggests that agricultural value added per worker contributes significantly to reducing rural poverty in Africa. On the other hand, food production index and GDP per capita were more important factors in curbing urban and dollar poverty implying that non-farm poor tends to have a large food marginal propensity to consume (MPC). The insignificance of GDP per capita in dwindling rural poverty reflects that the reality that growth in other sector does not influence the livelihood of the rural poor farmers due to its subsistence nature. Finally, domestic credit to private sectors and institutions were significant in reducing all categories of poverty, with largest impact on rural poverty. It implies that development programmes targeted at enhancing agricultural productivity should encompass strategies for accessing credit in order to boost the asset base of rural farmer for a large scale commercial production. Also, appropriate macroeconomic policies and institutional quality needs to be enhanced to boost provision of social services, equitable land and credit access.