The valorization of olive pomace through the extraction of phenolic compounds at an industrial scale is influenced by several factors that can have a significant impact on the feasibility of this approach. These include the types and levels of phenolic compounds that are present, the impact that seasonal variation and cultivar type have on the phenolic compound content in both olive pomace and mill effluents and the technological approach used to process the olive crop. Chemical analysis of phenolic compounds was performed using an HPLC-diode-array detector (DAD)-qTOF system, resulting in the identification of 45 compounds in olive mill wastewater and pomace, where secoiridoids comprised 50–60% of the total phenolic content. This study examined three different factors that could impact the phenolic compound content of these processing streams, including cultivar types typically grown on local farms in Slovenia, the type of downstream processing used and seasonality effects. Olive crop varieties sourced from local farms showed high variability, and the highest phenolic content was associated with the local variety “Istrska Belica”. During processing, the phenolic content was on average approximately 50% higher during two-phase decanting compared to three-phase decanting and the type of compound present significantly different. An investigation into the seasonal effects revealed that the phenolic content was 20% higher during the 2019 growing season compared to 2018. A larger sample size over additional growing seasons is required to fully understand the annual variation in phenolic compound content. The methods and results used in this study provide a basis for further analysis of phenolic compounds present in the European Union’s olive crop processing residues and will inform techno-economic modelling for the development of olive biorefineries in Slovenia.