This paper presents a vision based human-robotic interaction (HRI) framework for the modeling and localization of industrial objects typically found in an assembly task. Automating robotic vision for complicated industrial objects is an important, yet still difficult task, especially in the stage of extracting object features. To tackle this specific problem, we have developed a new HRI system consisting of an off-line vision model acquisition, in which the object's salient features are acquired through a human-in-the-loop approach. Subsequently, two feature extraction algorithms; region-growing and edge-grouping, are applied to the model development through collaboration between the human and robot. Finally, using a Kalman filter estimation with a proper ellipse representation, our object localization system generates ellipse hypotheses by grouping edge fragments in the scene, driven by the acquired vision model of objects. The proposed system is validated by experiments using actual industrial objects for both HRI-based object modeling and automated object localization.