This paper proposes two hybrid optimization methods based on Harmony Search algorithm (HS) and two different nature-inspired metaheuristic algorithms. In the first contribution, the combination was between the Improved Harmony Search (IHS) and the Particle Swarm Optimization (PSO). The second contribution merged the IHS with the Differential Evolution (DE) operators. The basic idea of hybridization was to ameliorate all the harmony memory vectors by adapting the PSO velocity or the DE operators in order to increase the convergence speed. The new algorithms (IHSPSO and IHSDE) have been compared to the IHS, DE, PSO and some other algorithms like DHS and HSDM. The DHS and HSDM are two existing algorithms, which use different hybridization concepts between HS and DE. All of these algorithms have been evaluated by different test Benchmark functions. The results demonstrated that the hybrid algorithm IHSDE have the better convergence speed into the global optimum than the IHSPSO and the standard IHS, DE and PSO.