Abstract Large quantities of mismanaged plastic waste are polluting and threatening the health of the blue planet. As such, vast amounts of this plastic waste found in the oceans originates from land. It finds its way to the open ocean through rivers, waterways and estuarine systems. Here we present a novel machine learning algorithm based on convolutional neural networks (CNNs) that is capable of detecting and quantifying floating and washed ashore plastic litter. The aquatic plastic litter detection, classification and quantification system (APLASTIC-Q) was developed and trained using very high geo-spatial resolution imagery (∼5 pixels cm−1 = 0.002 m pixel−1) captured from aerial surveys in Cambodia. APLASTIC-Q was made up of two machine learning components (i) plastic litter detector (PLD-CNN) and (ii) plastic litter quantifier (PLQ-CNN). PLD-CNN managed to categorize targets as water, sand, vegetation and plastic litter with an 83% accuracy. It also provided a qualitative count of litter as low or high based on a thresholding approach. PLQ-CNN further distinguished and enumerated the litter items in each of the classes defined as water bottles, Styrofoam, canisters, cartons, bowls, shoes, polystyrene packaging, cups, textile, carry bags small or large. The types and amounts of plastic litter provide benchmark information that is urgently needed for decision-making by policymakers, citizens and other public and private stakeholders. Quasi-quantification was based on automated counts of items present in the imagery with caveats of underlying object in case of aggregated litter. Our scientific evidence-based machine learning algorithm has the prospects of complementing net trawl surveys, field campaigns and clean-up activities for improved quantification of plastic litter. APLASTIC-Q is a smart algorithm that is easy to adapt for fast and automated detection as well as quantification of floating or washed ashore plastic litter from aerial, high-altitude pseudo satellites and space missions.