We propose a new class of univariate non‐stationary time series models, using the framework of modulated time series, which is appropriate for the analysis of rapidly evolving time series as well as time series observations with missing data. We extend our techniques to a class of bivariate time series that are isotropic. Exact inference is often not computationally viable for time series analysis, and so we propose an estimation method based on the Whittle likelihood, a commonly adopted pseudo‐likelihood. Our inference procedure is shown to be consistent under standard assumptions, as well as having considerably lower computational cost than exact likelihood in general. We show the utility of this framework for the analysis of drifting instruments, an analysis that is key to characterizing global ocean circulation and therefore also for decadal to century‐scale climate understanding.