The phytoplankton community composition, structure, and biomass were investigated under stratified and oligotrophic conditions during summer for three consecutive years in the Mediterranean Sea. Our results reveal that the phytoplankton community structure was strongly influenced by vertical stratification. The thermocline separated two different phytoplankton communities in the two layers of the euphotic zone, characterized by different nutrient and light availability. Picoplankton dominated in terms of abundance and biomass at all the stations sampled and throughout the photic zone. However, the structure of the picoplanktonic community changed with depth, with Synechococcus and heterotrophic prokaryotes dominating in surface waters down to the base of the thermocline, and Prochlorococcus and picoeukaryotes contributing relatively more to the community in the deep chlorophyll maximum (DCM). Light and nutrient availability also influenced the communities at the DCM layer. Prochlorococcus prevailed in deeper DCM waters characterized by lower light intensities and higher picophytoplankton abundance was related to lower nutrient concentrations at the DCM. Picoeukaryotes were the major phytoplankton contributors to carbon biomass at surface (up to 80%) and at DCM (more than 40%). Besides, contrarily to the other phytoplankton groups, picoeukaryotes cell size progressively decreased with depth. Our research shows that stratification is a major factor determining the phytoplankton community structure; and underlines the role that picoeukaryotes might play in the carbon flux through the marine food web, with implications for the community metabolism and carbon fate in the ecosystem.