Recent experiments have revealed ultrastrong coupling between light and matter as a promising avenue for modifying material properties, such as electrical transport, chemical reaction rates, and even superconductivity. Here, we explore (ultra)strong coupling as a means for manipulating the optical response of metamaterials based on ensembles of constituent units individually in the ultrastrong coupling regime. We develop a framework based on linear response for quantum electrodynamical systems to study how light-matter coupling affects the optical response. We begin by applying this framework to find the optical response of a two-level emitter coupled to a single cavity mode, which could be seen as a "meta-atom" of a metamaterial built from repeated units of this system. We find optical behaviors ranging from that of a simple two-level system (Lorentz-oscillator) to effectively transparent, as the coupling goes from the weak to deep strong coupling regimes. We explore a one-dimensional chain of these meta-atoms, demonstrating the tunability of its optical behavior. Our scheme may ultimately provide a framework for designing new metamaterials with low-loss, highly-confined modes, as well as tunable (single-photon) nonlinearities.