Left (or right) Leibniz algebras endowed with symmetric non-degenerate and associative bilinear forms (called quadratic Leibniz algebras) are investigated. In particular, we prove that left (resp. right) Leibniz algebras that carry this structure are also right (resp. left) Leibniz algebras. Moreover, we construct several examples of this type of algebras. Next, we prove that any solvable quadratic Leibniz algebra is a T*-extension (see M. Bordemann, Nondegenerate associative bilinear forms on nonassociative algebras, Acta Math. Univ. Com. LXIV 2 (1997) 151--201) of a solvable Lie algebra in the category of Leibniz algebras. In addition, we reduce the study of quadratic Leibniz algebras to that of quadratic Lie algebras by introducing some extensions of Leibniz algebras. Finally, we give an inductive description of quadratic Leibniz algebras by using T*-extensions and double extensions (central extension followed by generalized semi-direct product).