This paper presents a new approach to learning a compliance control law for robotic assembly tasks. In this approach, a task performance index of assembly operations is defined and the adaptive reinforcement learning algorithm [1] is applied for real-time learning. A simple box palletizing task is used as an example, where a robot is required to move a rectangular part to the corner of a box. In the experiment, the robot is initially provided with only predetermined velocity command to follow the nominal trajectory. However, at each attempt, the box is randomly located and the part is randomly oriented within the grasp of the end-effector. Therefore, compliant motion control is required to guide the part to the corner of the box while avoiding excessive reaction forces caused by the collision with a wall. After repeating failures in performing the task, the robot can successfully learn force feedback gains to modify its nominal motion. Our results show that the new learning method can be used to learn a compliance control law effectively.