This paper deals with the detection of orthogonal vanishing points. The aim is to efficiently cope with the clutter edges in real-life images and to determine the camera orientation in the Manhattan world reliably. We are using a modified scheme of the Cascaded Hough Transform where only one Hough space is accumulated – the space of the vanishing points. The parameterization of the vanishing points – the “diamond space” – is based on the PClines line parameterization and it is defined as a mapping of the whole real projective plane to a finite space. Our algorithm for detection of vanishing points operates directly on edgelets detected by an edge detector, skipping the common step of grouping edges into straight lines or line segments. This decreases the number of configuration parameters and reduces the complexity of the algorithm. Evaluated on the York Urban DB, our algorithm yields 98.04 % success rate at 10◦ angular error tolerance, which outperforms comparable existing solutions. Our parameterization of vanishing points is in all aspects linear; it involves no goniometric or other non-linear operations and thus it is suitable for implementation in embedded chips and circuitry. The iterative search scheme allows for finding orthogonal triplets of vanishing points with high accuracy and low computational costs. At the same time, our approach can be used without the orthogonality constraint.