ABSTRACTThe present work focuses on the investigation of the thermal and dielectric properties of composites obtained by surface‐modified hexagonal boron nitride (hBN) and ring‐opening metathesis polymerization (ROMP) based polymer. A new kind of high performance composites was developed based on using amino silane functionalized hBN (AS‐hBN) and bromine functional group possessing homo and copolymers synthesized via ROMP pathway. Aminosilane capped boron nitride (BN) and bromine bearing polymer backbone enhance the interaction between filler and the polymer chains. The effects of surface‐modified BN (AS‐hBN) and its content on the dielectric properties, and thermal resistance of composites, are systematically investigated and discussed. The resultant composites possess high electrical break over voltages. While all of the ROMP‐based films exhibit low ɛ′ value in a wide frequency range, in the case of the composite with 20% AS‐hBN and poly(bromooxanorbornene‐co‐cyclooctadiene) (ROMP‐BN‐6) displays very low dielectric constants in around 1.5 up to 1 MHz at 20 °C. This value is significantly lower than that of common polymer dielectrics, which is usually in the range of 3–6. Besides the lowest dielectric constant of ROMP‐BN‐6, it has also the smallest dielectric loss tangent even at high temperatures. Tan δ of ROMP‐BN‐6 is 0.003 and 0.0067 at 10 Hz–1 MHz at 20 °C, respectively. Thermal stability of polymers was also improved by introducing surface‐modified hBN. Polymers bearing 20% AS‐hBN are highly thermally stable up to ∼350 °C and gave 25% char yield at 800 °C. © 2017 Wiley Periodicals, Inc. J. Appl. Polym. Sci. 2018, 135, 45658.