The existing agriculture practices faced many challenges and fail to address some of the most critical needs of the growing population. Food insecurity, high initial cost of smart farming, severe farm labor shortage worldwide, economic, social, and political crises related to famines, poverty, climate change, and the technology focus of Agriculture 4.0 calls for rethinking the agriculture paradigm. Moreover, the idea of Society 5.0 promoted by Japanese government triggered many position reactions from policymakers, governments, private institutions, academicians, and researchers. The idea of human centered society where individuals live their lives to the fullest with shared vision of happiness, social harmony, sustainability, and resilience recently caught scholars’ attention. Several researchers investigated the society 5.0 and its critical components including Agriculture 5.0. Agriculture 5.0 not only could be leveraged to address many existing issues, but could become a major driving force for achieving Society 5.0’s goals. This paper follows a systematic literature review approach to investigate the major drivers, enabling cutting-edge technologies, various opportunities and challenges for developing, adopting, and implementation Agriculture 5.0. It also highlighted the overall and holistic architectural framework based on 12 layers of Agriculture 5.0 paradigm. Though Agriculture 5.0 is promising with many opportunities, such as creating new job opportunities for young generations, and boosting mass customization, it will face many potential challenges. Some challenges include cybersecurity and privacy issues, difficulties for an effective legal, regulatory and compliance system due to high automation and mass personalization, standardization issues, and adapting agricultural production strategies and models to constantly changing customer preferences.