This two-part paper is concerned with stability analysis of linear systems subject to parameter variations, of which linear time-invariant delay systems are of particular interest. We seek to characterize the asymptotic behavior of the characteristic zeros of such systems. This behavior determines, for example, whether the imaginary zeros cross from one half plane into another, and hence plays a critical role in determining the stability of a system. In Part I of the paper we develop necessary mathematical tools for this study, which focuses on the eigenvalue series of holomorphic matrix operators. While of independent interest, the eigenvalue perturbation analysis has a particular bearing on stability analysis and, indeed, has the promise to provide efficient computational solutions to a class of problems relevant to control systems analysis and design, of which time-delay systems are a notable example.