While solar power applications are growing rapidly worldwide, information about solar energy availability, its characteristics and the factors that affect it are essential. Among other parameters, a reference spectrum (ASTMG-173-03) is adopted, relying on Standard Test Conditions (STC), under which Photovoltaic (PV) devices are evaluated. However, these rigorously defined conditions can vary considerably from realistic environmental conditions. The objective of the present work is to assess the impact of the variability of atmospheric composition on the spectral distribution of the incident solar spectral irradiance (SSI) and, therefore, its implication on various PV materials performance. Ground-based measurements of global horizontal SSI have been conducted using a Precision Spectroradiometer (PSR) in the framework of the ASPIRE (Atmospheric parameters affecting SPectral solar IRradiance and solar Energy) project in Athens, Greece. The gathered data in combination with spectrally resolved radiative transfer under clear-sky conditions contributed to the investigation of the atmospheric variables that attenuate irradiance (e.g., aerosols). In addition, since PV modules’ spectral absorptivity differs according to the semiconductor material used, the impact of the above-mentioned spectral features on PV performance has been investigated in order to estimate the spectral impact between the theoretical and outdoor conditions on the yield of different PV technologies. Overall, the results denote that smoke has a more significant effect than dust, while the effect on various technologies varies. The highest deviation compared to the STC was observed in the case of a-Si, reaching an absolute difference of 45% in the case of smoke particles in the atmosphere, while the maximum deviation between the different technologies reached approximately 7%.