A new adaptive Differential Evolution algorithm called EWMA-DE is proposed. In original Differential Evolution algorithm three different control parameter values must be pre-specified by the user a priori; Population size, crossover constant and mutation scale factor. Choosing good parameters can be very difficult for the user, especially for the practitioners. In the proposed algorithm the mutation scale factor is adapted using a novel exponential moving average based mechanism, while the other control parameters are kept fixed as in standard Differential Evolution. The algorithm was initially evaluated by using the set of 25 benchmark functions provided by CEC2005 special session on real-parameter optimization and compared with the results of standard DE/rand/1/bin version. Results turned out to be rather promising; EWMA-DE outperformed the original Differential Evolution in majority of tested cases, which is demonstrating the potential of the proposed adaptation approach.