According to a recent report by the World Health Organization (WHO), the countries which still have limited access to water for drinking purposes are mainly those in the Sub-Saharan region. In this context, the current study provides an overview of the quality of surface water and groundwater in rural and peri-urban areas of the Republic of South Africa (RSA) and Mozambique (MZ) in terms of concentrations of conventional pollutants, inorganic chemicals, microorganisms, and micropollutants. Their values were compared with the drinking water standards available for the two countries. Regarding surface water, it was found that microorganisms occur at high concentrations; nickel (RSA) and boron (MZ) are other critical parameters. Regarding groundwater, arsenic and lead (RSA) and boron, sodium, and chloride (MZ) are the main critical substances. With regard to micropollutants, their surface water concentrations are much higher than those in European rivers. The highest values were for ibuprofen, acetylsalicylic acid, clozapine, and estriol. Suitable treatment is necessary to produce safe water depending on the main critical pollutants but, at the same time, action should be taken to improve wastewater treatment in rural areas to improve and safeguard surface water bodies and groundwater which are sources for drinking needs.