Accurate geolocation in GNSS-denied mountainous environments remains a significant challenge for navigation and field operations. While vision-based methods using terrain features like skylines offer a promising passive solution, their performance is often hindered by unknown camera orientation, low feature discriminability across vast areas, and severe degradation under nighttime conditions. To address these limitations, this paper presents a novel visual geolocation framework that synergistically couples ephemeris-based orientation correction with multi-scale skyline feature matching. First, an Edge-Enhanced SegFormer-B3 (EESFormer-B3) network is developed for robust skyline extraction from query images, where a lightweight Edge Gate mechanism is integrated to refine boundary precision, particularly under low-light scenarios. Second, we propose a camera heading estimation method utilizing solar and lunar ephemeris, eliminating reliance on external heading sensors and drastically reducing the matching search space. Third, for matching, a hierarchical composite feature is designed, encoding the skyline's global shape trends, local directional variations, and dominant structural distributions to ensure robustness against repetitive terrain patterns. Extensive experiments were conducted in a complex 700 km² mountainous region. The results demonstrate that our method achieves a localization success rate of 82% (daytime) and 76% (nighttime) within a 250-meter error threshold. The ephemeris-based correction alone accelerates the matching process by over 30 times compared to an unconstrained orientation search. This work provides a reliable, efficient, and illumination-invariant geolocation solution, with strong potential for applications in autonomous navigation, search and rescue, and other field missions in GNSS-compromised natural environments.