In this paper, a novel neural network (NN) adaptive dynamic programming (ADP) control scheme for distributed parameter systems (DPS) governed by parabolic partial differential equations (PDE) is introduced in the presence of control constraints and unknown system dynamics. First, Galerkin method is utilized to develop a relevant reduced order system which captures the dominant dynamics of the DPS. Subsequently, a novel control scheme is proposed over finite horizon by using NN ADP. To relax the requirement of system dynamics, a novel NN identifier is developed. More-over, a second NN is proposed to estimate online the time-varying non-quadratic value function from the Hamilton-Jacobi-Bellman (HJB) equation. Subsequently, by using the identifier and the value function estimator, the optimal control input that inherently lies in actuation limits is obtained. A local uniform ultimate boundedness (UUB) of the closed-loop system is verified by using standard Lyapunov theory. The performance of proposed control scheme and effects of its design parameters are successfully verified by simulation on a diffusion reaction process.