Tomato (Solanum lycopersicum L.) belongs to the Solanaceae family and is the second most important fruit or vegetable crop next to potato (Solanum tuberosum L.). It is cultivated for fresh fruit and processed products. Tomatoes contain many health-promoting compounds including vitamins, carotenoids, and phenolic compounds. In addition to its economic and nutritional importance, tomatoes have become the model for the study of fleshy fruit development. Tomato is a climacteric fruit and dramatic metabolic changes occur during its fruit development. In this review, we provide an overview of our current understanding of tomato fruit metabolism. We begin by detailing the genetic and hormonal control of fruit development and ripening, after which we document the primary metabolism of tomato fruits, with a special focus on sugar, organic acid, and amino acid metabolism. Links between primary and secondary metabolic pathways are further highlighted by the importance of pigments, flavonoids, and volatiles for tomato fruit quality. Finally, as tomato plants are sensitive to several abiotic stresses, we briefly summarize the effects of adverse environmental conditions on tomato fruit metabolism and quality.