Incorporating prior knowledge (PK) into learning methods is an effective means to improve learning performance. The consistency and error theories of PK-based methods, which are of great theoretical importance, are still far from well established. Concentrating on the PK-based kernel regression, this paper proposes a methodology of analyzing the consistency and error. This methodology converts the specific methods firstly to a unified optimization problem and then to a unified solution expression, and a general consistency and error analysis tool is proposed and applied. A few examples are given to illustrate the analysis procedure.