One investigates a mathematical programming problem with constraints in the form of inequalities and methods for its solution, based on the use of a nondifferentiable penalty function. Special attention has been given to the problem of the selection of a finite penalty parameter, ensuring the non-local convergence to the solution. It is shown that, as far as convergence is concerned, the constructed algorithms coincide with the known locally convergent algorithms of the conditional optimization with a linear approximation of the constraints, such as first-order methods (linearization, successive linearization methods), Newton's method, methods with accelerated convergence.