publication . Preprint . Conference object . 2017

A Cost-based Optimizer for Gradient Descent Optimization

Zoi Kaoudi; Jorge-Arnulfo Quiane-Ruiz; Saravanan Thirumuruganathan; Sanjay Chawla; Divy Agrawal;
Open Access English
  • Published: 27 Mar 2017
Abstract
As the use of machine learning (ML) permeates into diverse application domains, there is an urgent need to support a declarative framework for ML. Ideally, a user will specify an ML task in a high-level and easy-to-use language and the framework will invoke the appropriate algorithms and system configurations to execute it. An important observation towards designing such a framework is that many ML tasks can be expressed as mathematical optimization problems, which take a specific form. Furthermore, these optimization problems can be efficiently solved using variations of the gradient descent (GD) algorithm. Thus, to decouple a user specification of an ML task f...
Subjects
free text keywords: Computer Science - Databases, Optimization problem, Database, computer.software_genre, computer, Mathematical optimization, Gradient descent, Data mining, Operator (computer programming), Big data, business.industry, business, Computer science
25 references, page 1 of 2

[1] Apache Mahout: Scalable machine learning and data mining. http://mahout.apache.org/.

[2] Machine Learning Library (MLlib). http://spark.apache.org/mllib/.

[3] M. Abadi et al. TensorFlow: A System for Large-Scale Machine Learning. In OSDI, pages 265{283, 2016.

[4] D. Agrawal et al. Rheem: Enabling multi-platform task execution. In SIGMOD, 2016.

[5] D. Agrawal et al. Road to Freedom in Big Data Analytics. In EDBT, 2016.

[6] S. Ben-David and S. Shalev-Shwartz. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.

[7] D. P. Bertsekas. Nonlinear programming. chapter 1.3. Athena scienti c Belmont, 1999.

[8] M. Boehm et al. SystemML: Declarative Machine Learning on Spark. PVLDB, 9(13):1425{1436, 2016.

[9] M. Boehm, S. Tatikonda, B. Reinwald, P. Sen, Y. Tian, D. R. Burdick, and S. Vaithyanathan. Hybrid Parallelization Strategies for Large-scale Machine Learning in SystemML. PVLDB, 7(7):553{564, Mar. 2014.

[10] L. Bottou. Stochastic Gradient Descent Tricks. In Neural Networks: Tricks of the Trade. 2012. [OpenAIRE]

[11] O. Bousquet and L. Bottou. The tradeo s of large scale learning. In NIPS, pages 161{168, 2008.

[12] X. Feng, A. Kumar, B. Recht, and C. Re. Towards a Uni ed Architecture for in-RDBMS Analytics. In SIGMOD, pages 325{336, 2012.

[13] J. M. Hellerstein, C. Re, F. Schoppmann, D. Z. Wang, E. Fratkin, A. Gorajek, K. S. Ng, C. Welton, X. Feng, K. Li, and A. Kumar. The MADlib Analytics Library or MAD Skills, the SQL. PVLDB, 5(12):1700{1711, 2012. [OpenAIRE]

[14] B. Huang, S. Babu, and J. Yang. Cumulon: Optimizing Statistical Data Analysis in the Cloud. In SIGMOD, 2013.

[15] R. Johnson and T. Zhang. Accelerating Stochastic Gradient Descent using Predictive Variance Reduction. In NIPS, 2013.

25 references, page 1 of 2
Abstract
As the use of machine learning (ML) permeates into diverse application domains, there is an urgent need to support a declarative framework for ML. Ideally, a user will specify an ML task in a high-level and easy-to-use language and the framework will invoke the appropriate algorithms and system configurations to execute it. An important observation towards designing such a framework is that many ML tasks can be expressed as mathematical optimization problems, which take a specific form. Furthermore, these optimization problems can be efficiently solved using variations of the gradient descent (GD) algorithm. Thus, to decouple a user specification of an ML task f...
Subjects
free text keywords: Computer Science - Databases, Optimization problem, Database, computer.software_genre, computer, Mathematical optimization, Gradient descent, Data mining, Operator (computer programming), Big data, business.industry, business, Computer science
25 references, page 1 of 2

[1] Apache Mahout: Scalable machine learning and data mining. http://mahout.apache.org/.

[2] Machine Learning Library (MLlib). http://spark.apache.org/mllib/.

[3] M. Abadi et al. TensorFlow: A System for Large-Scale Machine Learning. In OSDI, pages 265{283, 2016.

[4] D. Agrawal et al. Rheem: Enabling multi-platform task execution. In SIGMOD, 2016.

[5] D. Agrawal et al. Road to Freedom in Big Data Analytics. In EDBT, 2016.

[6] S. Ben-David and S. Shalev-Shwartz. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.

[7] D. P. Bertsekas. Nonlinear programming. chapter 1.3. Athena scienti c Belmont, 1999.

[8] M. Boehm et al. SystemML: Declarative Machine Learning on Spark. PVLDB, 9(13):1425{1436, 2016.

[9] M. Boehm, S. Tatikonda, B. Reinwald, P. Sen, Y. Tian, D. R. Burdick, and S. Vaithyanathan. Hybrid Parallelization Strategies for Large-scale Machine Learning in SystemML. PVLDB, 7(7):553{564, Mar. 2014.

[10] L. Bottou. Stochastic Gradient Descent Tricks. In Neural Networks: Tricks of the Trade. 2012. [OpenAIRE]

[11] O. Bousquet and L. Bottou. The tradeo s of large scale learning. In NIPS, pages 161{168, 2008.

[12] X. Feng, A. Kumar, B. Recht, and C. Re. Towards a Uni ed Architecture for in-RDBMS Analytics. In SIGMOD, pages 325{336, 2012.

[13] J. M. Hellerstein, C. Re, F. Schoppmann, D. Z. Wang, E. Fratkin, A. Gorajek, K. S. Ng, C. Welton, X. Feng, K. Li, and A. Kumar. The MADlib Analytics Library or MAD Skills, the SQL. PVLDB, 5(12):1700{1711, 2012. [OpenAIRE]

[14] B. Huang, S. Babu, and J. Yang. Cumulon: Optimizing Statistical Data Analysis in the Cloud. In SIGMOD, 2013.

[15] R. Johnson and T. Zhang. Accelerating Stochastic Gradient Descent using Predictive Variance Reduction. In NIPS, 2013.

25 references, page 1 of 2
Powered by OpenAIRE Open Research Graph
Any information missing or wrong?Report an Issue
publication . Preprint . Conference object . 2017

A Cost-based Optimizer for Gradient Descent Optimization

Zoi Kaoudi; Jorge-Arnulfo Quiane-Ruiz; Saravanan Thirumuruganathan; Sanjay Chawla; Divy Agrawal;