This dataset supports the research study “TEMSET-24K: Densely Annotated Dataset for Indexing Multipart Endoscopic Videos using Surgical Timeline Segmentation” by Bilal et al. TEMSET-24K introduces a comprehensive taxonomy for Transanal Endoscopic Microsurgery (TEMS), capturing detailed surgical timelines through annotated phase, task, and action triplets. This structured approach was used by experienced surgeons to annotate colorectal cancer surgery videos, providing robust data for training surgical workflow analysis algorithms. Comprising 24,306 densely annotated microclips, this dataset is one of the largest collections released for advancing surgical workflow research. The dataset has significant translational potential for supporting clinical decisions and improving operational efficiency in clinical settings.