In the quest for machines that are able to learn, reinforcement learning (RL) is found to be an appealing learning methodology. A known problem in this learning method, however is that it takes too long before the robot learns to associate suitable situation-action pairs. Due to this problem, RL has remained applicable only to simple tasks and discrete environment. To accelerate the learning process to a level required by real robot tasks, the traditional learning architecture has to be modified. We propose a modified reinforcement based robot skill acquisition and adaptation architecture. The architecture has two components: a bias and a learning components. The bias component imparts to the learner coarse a priori knowledge about the task. Subsequently, the learner refines the acquired actions through reinforcement learning. We have validated the architecture and the learning algorithm on a simulated TRC mobile robot for a goal reaching task.