The heritability of borderline personality (BP) features has been established in multiple twin and family studies. Using data from the borderline subscale of the Personality Assessment Inventory Borderline Features Scale (PAI-BOR) collected in two Dutch cohorts (N=7125), the Netherlands Twin Register and The Netherlands Study of Depression and Anxiety, we show that heritability of the PAI-BOR total score using genome-wide single-nucleotide polymorphism (SNPs) is estimated at 23%, and that the genetic variance is substantially higher in affect instability items compared with the other three subscales of the PAI-BOR (42.7% vs non-significant estimates for self-harm, negative relations and identity problems). We present results from a first genome-wide association study of BP features, which shows a promising signal on chromosome 5 corresponding to SERINC5, a protein involved in myelination. Reduced myelination has been suggested as possibly having a role in the development of psychiatric disorders characterized by lack of social interaction. The signal was confirmed in a third independent Dutch cohort drawn from the Erasmus Rucphen Family study (N=1301). Our analyses were complemented by investigating the heterogeneity that was implied by the differences in genetic variance components in the four subscales of the PAI-BOR. These analyses show that the association of SNPs tagging SERINC5 differs substantially across the 24 items of the PAI-BOR. Further, using reverse regression we showed that the effects were present only in subjects with higher scores on the PAI-BOR. Taken together, these results suggest that future genome-wide analyses can benefit substantially by taking into account the phenotypic and genetic heterogeneity of BP features.