The establishment and maintenance of proper gene expression patterns is essential for stable cell differentiation. Using unsupervised learning techniques, chromatin states have been linked to discrete gene expression states, but these models cannot predict continuous gene expression levels, nor do they reveal detailed insight into the chromatin-based control of gene expression. Here, we employ regularized regression techniques to link, in a quantitative manner, binding profiles of chromatin proteins to gene expression levels and promoter-proximal pausing of RNA polymerase II in Drosophila melanogaster on a genome-wide scale. We apply stability selection to reliably detect interactions of chromatin features and predict several known, suggested, and novel proteins and protein pairs as transcriptional activators or repressors. Our integrative analysis reveals new insights into the complex interplay of transcriptional regulators in the context of gene expression. Supplementary Material is available at www.libertonline.com/cmb.