Abstract To improve on-site earthquake early warning for peak ground velocity (PGV), we leverage a machine learning approach. We propose a novel attention-based transformer architecture to address this challenging problem. A series of comparisons with other methods, including the traditional peak P-wave displacement amplitude approach and long short-term memory neural networks, is conducted. In addition, we demonstrate that the influence of building effects can be mitigated by incorporating station corrections to peak values in the seismograms as additional features during training. Finally, we discuss how the shape of the label can serve as a proxy to indicate the reliability of PGV determination within the first few seconds after the arrival time.