Understanding patterns of ecological specialization, the processes underlying niche partitioning and how they translate into the structure of interaction networks is a persistent challenge in ecology. Advances on this regard are limited by the prevalent focus on single spatial scales, lack of tests of the mechanisms underlying specialization, and scarce investigation of some types of interactions. Here we investigated the patterns of interaction between plants and florivores (flower‐breeding drosophilids, FBD) at species‐ and community‐level and at local and regional scales, and tested the relative importance of multiple potential drivers of frequencies of interactions in a local network. First, based on a year‐round collection of 42 322 flowers belonging to 82 plant species, we investigated species specialization and network structure and tested whether frequencies of interactions were related to plant–consumer temporal overlap and resource availability. Second, we built a regional florivore–plant meta‐network for the Neotropical region and tested its structure for nestedness, modularity and overall specialization. Our findings revealed that although FBD species span a broad range of degrees of specialization, most species were highly specialized. At both local and regional scales, network structure was highly modular and non‐nested, presenting high complementary specialization. Moreover, phenological overlap between FBD and their hosts was the most influential driver of frequency of interactions, in comparison to abundance and traits. By describing the structure of FBD–plant networks, our results illustrate how a highly diverse and specialized system retains interaction patterns found in other types of interaction networks which are often driven by different processes. Furthermore, despite undersampling of interactions in the meta‐network caused by the lack of studies on this system and the high diversity of Neotropical FBD's, the high modularity and consistency of some clade–clade modules across spatial scales suggests the importance of evolutionary history and physiological constraints in shaping interactions in this system.