This study aimed to compare the acute effect of a proprioceptive exercise session and a non-specific exercise session on knee position sense, and the static and dynamic balance of athletes. Sixty male athletes (19.4 ± 1.2 years) participated in a within-subjects repeated-measures study. Knee position sense in closed kinetic chain, and static (BESS test) and dynamic balance (Y-balance test) were measured before and after two exercise sessions, consisting of 10 min of non-specific exercise in a cycle-ergometer or proprioceptive exercise with an unstable platform. Overall, both exercise sessions significantly improved knee position sense, BESS score, and YBT composite score, and no differences were detected between proprioceptive and non-specific sessions (knee position sense, −6.9 ± 65.2% vs. −11.5 ± 75.0%, p = 0.680; BESS, −19.3 ± 47.7% vs. −29.03 ± 23.5%, p = 0.121; YBT, 2.6 ± 2.7% vs. 2.2 ± 2.2%, p = 0.305). Twenty athletes did not improve knee position sense after the exercise session (non-responders). When analyzing only the exercise responders, both sessions improved knee position sense, but the improvement was greater after the proprioceptive exercise session (56.4 ± 25.6% vs. 43.8 ± 18.9%, p = 0.023). In conclusion, a single proprioceptive, as well as non-specific, exercise session increased knee position sense and balance. The proprioceptive exercise seems to be more effective in improving joint position sense when considering only athletes who respond to the intervention.