In this paper, we introduce and explore a new computing paradigm we call knowledge as a service, in which a knowledge service provider, via its knowledge server, answers queries presented by some knowledge consumers. The knowledge server's answers are based on knowledge models that may be expensive or impossible to obtain for the knowledge consumers. While this new paradigm of computing is promising, we must establish a solid foundation to ensure its utility. We focus on the security aspect of the paradigm, and particularly on the problem we call knowledge breaching attack, which may allow an adversary to recover the knowledge underlying a knowledge service. Without being able to adequately handling such an attack, the knowledge service providers would never have any economic incentives to develop such a paradigm. Unfortunately, this paper theoretically shows that any interesting knowledge is subject to the knowledge breaching attack, and empirically shows that some knowledge models could be breached after a very small number of queries (e.g., 0.2-]% portion of the domain). Thus we need to investigate technical means that can alleviate such powerful attacks (at least for most practical knowledge models).