Distributed Coordinate Descent Method for Learning with Big Data

Article, Preprint, Research English OPEN
Richtárik, Peter; Takáč, Martin;
  • Subject: cs.LG | Mathematics - Optimization and Control | stat.ML | Computer Science - Distributed, Parallel, and Cluster Computing | Statistics - Machine Learning | cs.DC | math.OC | Computer Science - Learning

In this paper we develop and analyze Hydra: HYbriD cooRdinAte descent method for solving loss minimization problems with big data. We initially partition the coordinates (features) and assign each partition to a different node of a cluster. At every iteration, each node... View more
  • References (20)
    20 references, page 1 of 2

    Bradley, J., Kyrola, A., Bickson, D., and Guestrin, C. Parallel coordinate descent for l1-regularized loss minimization. In ICML, 2011.

    Fercoq, O. Parallel coordinate descent for the AdaBoost problem. In ICMLA, 2013.

    Fercoq, O. and Richta´rik, P. Smooth minimization of nonsmooth functions with parallel coordinate descent methods. arXiv:1309.5885, 2013.

    Hsieh, C-J., Chang, K-W., Lin, C-J., Keerthi, S.S., , and Sundarajan, S. A dual coordinate descent method for large-scale linear SVM. In ICML, 2008.

    Lu, Z. and Xiao, L. On the complexity analysis of randomized block-coordinate descent methods. arXiv:1305.4723, 2013.

    Mukherjee, I., Singer, Y., Frongillo, R., and Canini, K. Parallel boosting with momentum. In ECML, 2013.

    Necoara, I., Nesterov, Yu., and Glineur, F. Efficiency of randomized coordinate descent methods on optimization problems with linearly coupled constraints. Technical report, 2012.

    Nesterov, Yu. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2):341-362, 2012.

    Nesterov, Yu. Gradient methods for minimizing composite objective function. Mathematical Programming, pp. 125-161, 2013.

    Richta´rik, P. and Taka´cˇ, M. Parallel coordinate descent methods for big data optimization. arXiv:1212.0873, 2012a.

  • Metrics
Share - Bookmark