We give an exposition of a unified study of categories with extra structure that arise in computer science and mathematics. We consider several examples of structures that arise, showing that with a precise formulation of the notion of category with algebraic structure, they are categories with algebraic structure. We then outline the central results and issues in the study of categories with algebraic structure, with an account of why those issues are of computational interest. We illustrate general theorems that yield substantial results in examples given by familiar structures. In particular, we explain known mathematics that supports the idea of a programming language being freely generated by basic data and specified algebraic structure. We then show in detail how the concept of algebraic structure may be used in defining new category theoretic structures by showing how it affected the precise formulation of premonoidal category, as has recently been proposed to account for contexts.