With the increasing global attention to environmental protection and the sustainable use of resources, waste classification has become a critical issue that needs urgent resolution in social development. Compared with the traditional manual waste classification methods, deep learning-based waste classification systems offer significant advantages. This paper proposes an innovative deep learning framework, Garbage FusionNet (GFN), aimed at tackling the waste classification challenge. GFN enhances classification performance by integrating the local feature extraction strengths of ResNet with the global information processing capabilities of the Vision Transformer (ViT). Furthermore, GFN incorporates the Pyramid Pooling Module (PPM) and the Convolutional Block Attention Module (CBAM), which collectively improve multi-scale feature extraction and emphasize critical features, thereby increasing the model’s robustness and accuracy. The experimental results on the Garbage Dataset and Trashnet demonstrate that GFN achieves superior performance compared with other comparison models.