AbstractOn open surfaces, the energy space of hypersingular operators is a fractional order Sobolev space of order 1/2 with homogeneous Dirichlet boundary condition (along the boundary curve of the surface) in a weak sense. We introduce a boundary element Galerkin method where this boundary condition is incorporated via the use of a Lagrangian multiplier. We prove the quasi‐optimal convergence of this method (it is slightly inferior to the standard conforming method) and underline the theory by a numerical experiment.The approach presented in this article is not meant to be a competitive alternative to the conforming method but rather the basis for nonconforming techniques like the mortar method, to be developed. © 2008 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq, 2009