Scene understanding is a foundational aspect of computer vision, aiming to emulate the intricate processes of the human visual system. This involves discerning the myriad of subtle cues embedded within the multifaceted visual environment and subsequently interpreting the visual narratives that unfold around us. Historically, much of the research in this domain has been concentrated on 2D or 2.5D scene understanding. These efforts, however, have often been hampered by the lack of comprehensive datasets and the limitations inherent in the available scene processing methodologies. In contrast, the research presented in this thesis delves into the realm of 3D Point-based Scene Understanding. This is a nascent yet burgeoning area within 3D visual recognition that, until now, has not been extensively investigated. By focusing on this innovative approach, the research seeks to extract and analyze scene structures that are of paramount significance, both from synthesized 3D computer-aided scenes and from scans of real-world 3D environments. The implications of this research are profound. Not only does it hold considerable promise for tasks that bridge 2D and 3D vision, such as scene matching and retrieval, but it also unveils new avenues for exploration in the domain of intelligent multimedia research. Given its practical applications, the insights gleaned from this study could be instrumental in revolutionizing various AI-driven sectors that rely heavily on scene interpretation. A prime example of this would be the industry of autonomous vehicles, where accurate and nuanced scene understanding is pivotal for safe and efficient navigation.