In mathematical applications, category theory remains a contentious issue, with enthusiastic fans and a skepticalmajority. In a muted form this split applies to the authors ofthis note. When we learned that the only mathematically soundfoundation of topological quantum computing in the literature isbased on category theory, the skeptical author suggested to "decategorize" the foundation. But we discovered, to our surprise, thatcategory theory (or something like it) is necessary for the purpose,for computational reasons. The goal of this note is to give a high-level explanation of that necessity, which avoids details and whichsuggests that the case of topological quantum computing is farfrom unique.