Visible Light Positioning (VLP) has been prevalent in providing high-precision localization systems in the past decade. However, the commercial availability or usage is still limited primarily due to the requirement of changing the existing lighting infrastructure. In this paper, we propose HueSense, an alternative technique to develop a passive VLP system by extracting light-emission intrinsic features, such as dominant colours present in the white LED light. The method can eliminate the need to change lighting-infrastructure, and only uses cheaper and power-efficient off-the-shelf hue sensors. Our experiments demonstrate that HueSense can achieve a location-mapping accuracy of 80.14% with a moving robot in uncontrolled lighting environments.