This dissertation presents new algorithms for identifying structure in large and complex datasets. The work focuses on clustering methods based on convex optimization, which group similar items by minimizing a mathematically well-behaved loss function. A key contribution is convex clustering through majorization–minimization (CCMM), an approach that achieves speeds several orders of magnitude faster than existing techniques and scales to datasets with over one million observations. The framework is extended to convex biclustering, which simultaneously clusters both observations and variables, with computational complexity that grows only linearly with dataset size.The dissertation further applies these ideas to Gaussian graphical modeling, aiming to estimate networks of variables with block-structured patterns. The resulting clusterpath estimator of the Gaussian graphical model (CGGM) employs an efficient coordinate descent algorithm and demonstrates strong performance in detecting clusters in both simulated and empirical settings.The final contribution is the classifier chain network for multi-label classification, a method that jointly estimates parameters to capture dependencies between multiple binary outcomes. Across diverse applications, the proposed methods deliver substantial gains in scalability, interpretability, and accuracy, offering practical tools for clustering, network estimation, and predictive modeling in high-dimensional data analysis.