Computational simulations have become indispensable in physics, materials science, and chemistry, enabling deep insights into material properties and molecular behavior. A central challenge in these fields is efficiently exploring the potential energy surface, which is essential for understanding and predicting system behaviors at the atomic level. This thesis aims to develop and enhance computational methods for efficient exploration of the potential energy surface through three key projects. Firstly, we introduce the symmetry penalty function, a novel approach for global optimization algorithms like Minima Hopping. By applying Pauling's rule of structural parsimony, the method biases the search towards structures with similar atomic environments, effectively reducing the configurational space and improving search efficiency. This approach led to significant computational speedups, achieving up to 25-fold faster discovery of ground states in silicon carbide and 63-fold in C$_{60}$ molecules. Secondly, we develop the mode-based finite difference method for vibrational frequency calculations. The traditional finite difference method in Cartesian coordinates is sensitive to noise in the forces, this is problematic because density functional theory calculations inherently have some noise in the forces. The mode-based method mitigates this by performing finite differences in mode space, reducing the impact of the noise on the forces and enhancing accuracy. Applied to Na$_{55}$ clusters and ethanol molecules, the mode-based method achieved mean relative errors up to two orders of magnitude lower than the Cartesian-based method. Thirdly, utilizing the advancement from our mode-based method for the vibrational analysis, we conducted an extensive global structure search on sodium clusters ranging from 50 to 370 atoms using the Minima Hopping algorithm with a Finnis-Sinclair potential. This study reveals the emergence of body-centered cubic structures in clusters of approximately 276 to 338 atoms, the first observation ...