The complexity of problems faced in many practical applications, inter alia occurring in natural, physical, and engineering science, is increasingly prevalent. However, until this very day, there exists no fully developed theory for grand challenges such as climate (change), disease modeling, or the unraveling of all functionalities of the human brain. One may remark, that despite the partly extensively varying context, these topicalities all may be classified as high-dimensional phenomena which evolve (nonlinearly) in time. Such, however, are typically confronted with one grand challenge: they are (yet) too complex to be understood by a first principles approach. The era of Big Data with the rise of memory capacities, processor frequencies, and further developed technology, introduces another strategy to be considered: The concept of data-driven techniques. Thus, instead of formulating complete theorems first, one may extract the most dominant system dynamics directly from the corresponding data. Though tempting in theory, one must be particularly aware of one challenge coming along with inflating data amounts, i.e., the growth of execution time. Thus, to overcome this obstacle, one may exploit optimization techniques of High-Performance Computing (HPC). The conflation of both, i.e., the development of a data-driven HPC technique, with the concrete example of the (Parallel) Dynamic Mode Decomposition, is discussed in this thesis.