The simulation of physical systems over long time horizons is a fundamental challenge in computational science, where standard numerical integrators often accumulate errors that lead to non-physical behavior, such as the violation of conservation laws. This paper investigates a class of machine learning models, termed structure-preserving neural networks, designed to overcome these limitations by embedding physical inductive biases directly into their architecture. Traditional neural networks, when used to model dynamical systems, often fail to generalize and exhibit poor long-term stability due to their inability to respect underlying physical symmetries. Structure-preserving models, such as Hamiltonian Neural Networks (HNNs) and Lagrangian Neural Networks (LNNs), learn the underlying energy function of a system rather than the forces directly. This approach ensures the conservation of energy by construction. Furthermore, by integrating these learned Hamiltonians or Lagrangians with symplectic integrators, these models can also preserve the geometric structure of the phase space, a critical property for stable long-term prediction of Hamiltonian systems. This work provides a comprehensive overview of these architectures, detailing how they leverage principles from classical mechanics to achieve superior performance. We discuss the theoretical foundations of these models, rooted in geometric numerical integration, and present a methodological framework for their implementation. By explicitly encoding conservation laws and geometric structures, these networks not only achieve remarkable accuracy and stability in long-term trajectory prediction but also produce more interpretable and generalizable models from sparse data, paving the way for their application in discovering and modeling complex physical phenomena.