We study best approximations to compact operators between Banach spaces and Hilbert spaces, from the point of view of Birkhoff-James orthogonality and semi-inner-products. As an application of the present study, some distance formulae are presented in the space of compact operators. The special case of bounded linear functionals as compact operators is treated separately and some applications to best approximations in reflexive, strictly convex and smooth Banach spaces are discussed. An explicit example is presented in ℓ p n \ell _p^{n} spaces, where 1 > p > ∞ , 1 > p > \infty , to illustrate the applicability of the methods developed in this article. A comparative analysis of the results presented in this article with the well-known classical duality principle in approximation theory is conducted to demonstrate the advantage in the former case, from a computational point of view.