Free Search (FS) algorithm is efficient in solving complex function optimization problems. Convergence of FS is analyzed in two different cases: continuous and discrete space. For continuous space, convergence of FS did not exist for all functions, such as functions containing singular points. However, taking advantage of measure theory, convergence of FS could be shown when functions satisfy continuous and Lipschitz conditions. For discrete space, making use of random process theory, convergence of FS was got in finite search space and FS was characterized by Markov property. Simulation of multi-model Shubert function is done. Compared with Genetic Algorithm (GA), FS is superior in convergence speed, accuracy and robustness. The analytic and experimental results on FS provide useful evidence for further understanding and properly tackling optimization problems of complex functions.