Since hash proof system (HPS) can be utilized to build versatilely cryptographic schemes, the study on realizing this cryptographic primitive has been a very active research area. With the increasing concerns on the huge progress in quantum computing, it urges cryptographers to explore the existence of quantum-resistant HPS schemes, such as the one relying on some lattice-based assumptions. However, most lattice-based HPS proposals are relatively inefficient (e.g., simply outputting one-bit key), even though lattice-based schemes can enjoy many advantageous features: worst-case to average-case reduction, resistance so far to quantum algorithms, and good asymptotic efficiency. Therefore, efficient HPS schemes based on lattice problems are deeply in demand. Through a comprehensive analysis, we found that some lattice-based HPS schemes can be rephrased as their corresponding key encapsulation mechanism (KEM) forms, which generally rely on diverse reconciliation mechanisms and directly imply key exchange protocols under lattice-based assumptions. In this paper, inspired by a novel reconciliation mechanism based on the learning with errors (LWE) problem, we first properly adapt this LWE-based reconciliation mechanism for arbitrary modulus. Then using this improved reconciliation mechanism, we propose an efficient LWE-based HPS scheme which can generate multiple encapsulated key bits and perform better in both computation and storage costs than other related results. Moreover, our proposed lattice-based HPS scheme can be also extended to identity-based and updatable settings for demonstrating its diverse applications.