The transport of cargo proteins to specific subcellular destinations is crucial for the different secretory and endocytic traffic pathways. One of the most important steps in maintaining the accuracy of this process is the recruitment of adaptor protein (AP) complexes to the membrane for recognizing and packaging cargo proteins into nascent vesicles. Adaptor protein complex 3 (AP-3) is a heterotetrametric complex implicated in the trafficking of cargo proteins from the trans-Golgi network (TGN) and/or endosomes to lysosomes or lysosome-related organelles (LROs). This complex is also involved in the biogenesis of synaptic vesicles (SVs) in neurons and of dense core vesicles (DCVs) in endocrine cells as well as in the recycling of receptors in immune cells and the regulation of planar cell polarity (PCP) proteins. Functional defects in AP-3 cause multiple abnormalities in cellular vesicle trafficking and related organelle function, leading to various disorders, such as Hermansky-Pudlak syndrome (HPS). However, the molecular mechanism underlying AP-3 has not been fully elucidated, and further investigations are needed to understand AP-3-mediated trafficking, its associated molecules and its related roles in inherited diseases. Here, we review the current understanding of AP-3 in cellular vesicle trafficking, especially focusing on mammalian systems.