We introduce a generalization of the celebrated ordinal pattern approach for the analysis of time series, in which these are evaluated in terms of their distance to ordinal patterns defined in a continuous way. This allows us to naturally incorporate information about the local amplitude of the data and to optimize the ordinal pattern(s) to the problem under study. This last element represents a novel bridge between standard ordinal analysis and deep learning, allowing the achievement of results comparable to the latter in real-world classification problems while also retaining the conceptual simplicity, computational efficiency, and easy interpretability of the former. We test this through the use of synthetic time series, generated by standard chaotic maps and dynamical models, data sets representing brain activity in health and schizophrenia, and the dynamics of delays in the European air transport system. We further show how the continuous ordinal patterns can be used to assess other aspects of the dynamics, like time irreversibility.