research product . 2017

The concrete distribution: A continuous relaxation of discrete random variables

Maddison, C; Mnih, A; Teh, Y;
  • Published: 24 Apr 2017
  • Country: United Kingdom
Abstract
The reparameterization trick enables optimizing large scale stochastic computation graphs via gradient descent. The essence of the trick is to refactor each stochastic node into a differentiable function of its parameters and a random variable with fixed distribution. After refactoring, the gradients of the loss propagated by the chain rule through the graph are low variance unbiased estimators of the gradients of the expected loss. While many continuous random variables have such reparameterizations, discrete random variables lack useful reparameterizations due to the discontinuous nature of discrete states. In this work we introduce CONCRETE random variables—C...
Read more
Related Organizations
Download from
Oxford University Research Archive
2017
Provider: Oxford University Research Archive
Powered by OpenAIRE Research Graph
Any information missing or wrong?Report an Issue