In the past 20 years, we have reached a broad understanding of many light-driven phenomena in nanoscale systems. The temporal dynamics of the excited states are instead quite challenging to explore, and, at the same time, crucial to study for understanding the origin of fundamental physical and chemical processes. In this review, we examine the current state and prospects of ultrafast phenomena driven by plasmons both from a fundamental and applied point of view. This research area is referred to as ultrafast plasmonics and represents an outstanding playground to tailor and control fast optical and electronic processes at the nanoscale, such as ultrafast optical switching, single photon emission, and strong coupling interactions to tailor photochemical reactions. Here, we provide an overview of the field and describe the methodologies to monitor and control nanoscale phenomena with plasmons at ultrafast timescales in terms of both modeling and experimental characterization. Various directions are showcased, among others recent advances in ultrafast plasmon-driven chemistry and multi-functional plasmonics, in which charge, spin, and lattice degrees of freedom are exploited to provide active control of the optical and electronic properties of nanoscale materials. As the focus shifts to the development of practical devices, such as all-optical transistors, we also emphasize new materials and applications in ultrafast plasmonics and highlight recent development in the relativistic realm. The latter is a promising research field with potential applications in fusion research or particle and light sources providing properties such as attosecond duration.