The paper investigates the behaviour of Wald's test (the maximum likelihood test statistic) when applied to hypothesis testing in one- parameter exponential families for one-sample problems. Conditions under which Wald's test is well behaved and conditions under which Wald's test may be misleading are derived, the main conclusion being that in general the test statistic should be used with caution in discrete probability models due to certain boundary problems. However, Wald's test may also be misleading in the continuous case if the upper tail of the distribution function is approximately proportional to \(t^{-1}e^{-\theta t}\) for some positive \(\theta\). An indication of how some of the results carry over to generalized linear models is also given.