We study the boundedness properties of pseudo-differential operators a(x, D) and their adjoints a(x, D)* with symbols in a certain vector-valued Besov space on Besov spaces B^s_{p,q} and Triebel spaces F^s_{p,q} (0 < p,q < \infty) . Applications are given to multiplication properties of Besov and Triebel spaces. We show that our results are best possible for both pseudo-differential estimates and multiplication. Denoting by (\cdot,\cdot) the duality between Besov and between Triebel spaces we derive general conditions under which (a(x, D)f, g) = (f a(x, D)*g) holds. This requires a precise definition of a(x, D)f and a(x, D)*f for f \in F^s_{p,q} and f \in B^s_{p,q} .