We present a Hamiltonian formulation for the linearized Vlasov-Maxwell system with a Maxwellian background distribution function. We discuss the geometric properties of the model at the continuous level, and how to discretize the model in the GEMPIC framework [1]. This method allows us to preserve the structure of the system at the semi-discrete level. To integrate the model in time, we employ a Poisson splitting and discuss how to integrate each subsystem separately. We test the model against the direct delta-f method, which is the non-geometric pendant of our model. The first test case is the weak Landau damping, where our model exhibits the same physical properties for short simulations, but enjoys better long-time stability and energy conservation due to its geometric construction. These advantages becomes even more pronounced for the simulation of Bernstein waves, our second test case, where the noise in the direct delta-f method washes out all features of the dispersion relation whereas our model is able to reproduce the full spectrum correctly. The model is implemented in the open-source Python library STRUPHY [2], [3].