Retromer (VPS26/VPS35/VPS29) is a highly conserved eukaryotic protein complex that localizes to endosomes to sort transmembrane protein cargoes into vesicles and elongated tubules. Retromer mediates retrieval pathways from endosomes to the trans-Golgi network in all eukaryotes and further facilitates recycling pathways to the plasma membrane in metazoans. In cells, retromer engages multiple partners to orchestrate the formation of tubulovesicular structures, including sorting nexin (SNX) proteins, cargo adaptors, GTPases, regulators, and actin remodeling proteins. Retromer-mediated pathways are especially important for sorting cargoes required for neuronal maintenance, which links retromer loss or mutations to multiple human brain diseases and disorders. Structural and biochemical studies have long contributed to the understanding of retromer biology, but recent advances in cryo-electron microscopy and cryo-electron tomography have further uncovered exciting new snapshots of reconstituted retromer structures. These new structures reveal retromer assembles into an arch-shaped scaffold and suggest the scaffold may be flexible and adaptable in cells. Interactions with cargo adaptors, particularly SNXs, likely orient the scaffold with respect to phosphatidylinositol-3-phosphate (PtdIns3P)-enriched membranes. Pharmacological small molecule chaperones have further been shown to stabilize retromer in cultured cell and mouse models, but mechanisms by which these molecules bind remain unknown. This review will emphasize recent structural and biophysical advances in understanding retromer structure as the field moves towards a molecular view of retromer assembly and regulation on membranes.