It is shown that the vacuum expectation values $W({C}_{1},\ensuremath{\cdots},{C}_{n})$ of products of the traces of the path-ordered phase factors $P\mathrm{exp}[ig\ensuremath{\oint}\ensuremath{\int}{{C}_{i}}^{}{\mathit{A}}_{\ensuremath{\mu}}(x)d{x}^{\ensuremath{\mu}}]$ are multiplicatively renormalizable in all orders of perturbation theory. Here ${\mathit{A}}_{\ensuremath{\mu}}(x)$ are the vector gauge field matrices in the non-Abelian gauge theory with gauge group $\mathrm{U}(N)$ or $\mathrm{SU}(N)$, and ${C}_{i}$ are loops (closed paths). When the loops are smooth (i.e., differentiable) and simple (i.e., non-self-intersecting), it has been shown that the generally divergent loop functions $W$ become finite functions $\stackrel{\ifmmode \tilde{}\else \~{}\fi{}}{W}$ when expressed in terms of the renormalized coupling constant and multiplied by the factors ${e}^{\ensuremath{-}KL({C}_{i})}$, where $K$ is linearly divergent and $L({C}_{i})$ is the length of ${C}_{i}$. It is proved here that the loop functions remain multiplicatively renormalizable even if the curves have any finite number of cusps (points of nondifferentiability) or cross points (points of self-intersection). If ${C}_{\ensuremath{\gamma}}$ is a loop which is smooth and simple except for a single cusp of angle $\ensuremath{\gamma}$, then ${W}_{R}({C}_{\ensuremath{\gamma}})=Z(\ensuremath{\gamma})\stackrel{\ifmmode \tilde{}\else \~{}\fi{}}{W}({C}_{\ensuremath{\gamma}})$ is finite for a suitable renormalization factor $Z(\ensuremath{\gamma})$ which depends on $\ensuremath{\gamma}$ but on no other characteristic of ${C}_{\ensuremath{\gamma}}$. This statement is made precise by introducing a regularization, or via a loop-integrand subtraction scheme specified by a normalization condition ${W}_{R}({\overline{C}}_{\ensuremath{\gamma}})=1$ for an arbitrary but fixed loop ${\overline{C}}_{\ensuremath{\gamma}}$. Next, if ${C}_{\ensuremath{\beta}}$ is a loop which is smooth and simple except for a cross point of angles $\ensuremath{\beta}$, then $\stackrel{\ifmmode \tilde{}\else \~{}\fi{}}{W}({C}_{\ensuremath{\beta}})$ must be renormalized together with the loop functions of associated sets ${{S}^{i}}_{\ensuremath{\beta}}={{{C}^{i}}_{1},\ensuremath{\cdots},{{C}^{i}}_{\mathrm{pi}}}$ ($i=2,\ensuremath{\cdots},I$) of loops ${{C}^{i}}_{q}$ which coincide with certain parts of ${C}_{\ensuremath{\beta}}\ensuremath{\equiv}{{C}_{1}}^{1}$. Then ${W}_{R}({{S}^{i}}_{\ensuremath{\beta}})={Z}^{\mathrm{ij}}(\ensuremath{\beta})\stackrel{\ifmmode \tilde{}\else \~{}\fi{}}{W}({{S}^{j}}_{\ensuremath{\beta}})$ is finite for a suitable matrix ${Z}^{\mathrm{ij}}(\ensuremath{\beta})$. Finally, for a loop with $r$ cross points of angles ${\ensuremath{\beta}}_{1},\ensuremath{\cdots},{\ensuremath{\beta}}_{r}$ and $s$ cusps of angles ${\ensuremath{\gamma}}_{1},\ensuremath{\cdots},{\ensuremath{\gamma}}_{s}$, the corresponding renormalization matrices factorize locally as ${Z}^{{i}_{1}{j}_{1}}({\ensuremath{\beta}}_{1})\ensuremath{\cdots}{Z}^{{i}_{r}{j}_{r}}({\ensuremath{\beta}}_{r})Z({\ensuremath{\gamma}}_{1})\ensuremath{\cdots}Z({\ensuremath{\gamma}}_{s})$.