Phosphatidylinositol 3-kinases (PI3Ks) are important therapeutic targets for the treatment of cancer, thrombosis, and inflammatory and immune diseases. The four highly homologous Class I isoforms, PI3Kα, PI3Kβ, PI3Kγ and PI3Kδ have unique, non-redundant physiological roles and as such, isoform selectivity has been a key consideration driving inhibitor design and development. In this review, we discuss the structural biology of PI3Ks and how our growing knowledge of structure has influenced the medicinal chemistry of PI3K inhibitors. We present an analysis of the available structure-selectivity-activity relationship data to highlight key insights into how the various regions of the PI3K binding site influence isoform selectivity. The picture that emerges is one that is far from simple and emphasizes the complex nature of protein-inhibitor binding, involving protein flexibility, energetics, water networks and interactions with non-conserved residues.