We study the use of approximate Lagrange multipliers in the stochastic subgradient method for the dual problem in constrained convex optimisation. The use of approximate Lagrange multipliers in the optimisation (instead of the true multipliers) is motivated by the fact that it is possible to accurately approximate some non-convex control problems as convex optimisations. For example, it is possible to solve certain stochastic discrete decision problems by solving a sequence of convex optimisations. We show how the analysis can be used in networking problems with queues, and present a wireless example that has constraints on how control actions can be selected which illustrates the power of the approach.