The rapidly developing quantum technologies and thermodynamics have put forward a requirement to precisely control and measure the temperature of microscopic matter at the quantum level. Many quantum thermometry schemes have been proposed. However, precisely measuring low temperature is still challenging because the obtained sensing errors generally tend to diverge with decreasing temperature. Using a continuous-variable system as a thermometer, we propose non-Markovian quantum thermometry to measure the temperature of a quantum reservoir. A mechanism to make the sensing error $δT$ scale with the temperature $T$ as the Landau bound $δT\simeq T$ in the full-temperature regime is discovered. Our analysis reveals that it is the quantum criticality of the total thermometer-reservoir system that causes this enhanced sensitivity. Efficiently avoiding the error-divergence problem, our result gives an efficient way to precisely measure the low temperature of quantum systems.