In our present study, we introduce and examine several new subclasses of analytic functions defined through the convolution operator HYλ,q, which is formulated using the classical error function. Our approach relies on the concept of quasi-subordination, a broad extension of subordination in geometric function theory. For each of these newly defined classes, we focus on deriving significant properties such as the upper bounds of the first few Taylor–Maclaurin coefficients of normalized series, evaluation of classical Fekete–Szegö functional, and calculating the upper bounds of Hankel and Toeplitz determinants for different orders. The combination of the proposed operator and the quasi-subordination framework offers a unified strategy for tackling these problems. Our findings also generalize various existing results in the field.