The application of artificial intelligence (AI) technology in the field of power systems and power electronic devices is increasingly prevalent. With massive datasets generated by a wide range of equipment, AI-based modeling is promising in the future of hardware-in-the-loop (HIL) emulation. This paper studies and improves the machine learning (ML) based modeling approach for power electronic devices, and the inferencer-in-the-loop (IIL) system is proposed together with optimized neural network (NN) models. The high-speed rail (HSR) microgrid, includes autotransformer rectifier unit subsystems (ATRUSs), energy storage subsystems (ESSs), two-level converter based permanent magnet synchronous motor (TLC PMSM) propulsion subsystems, and modular multilevel converter based induction motor (MMC-IM) propulsion subsystems, serves as study cases to demonstrate the adaptability of this approach. Finally, to show high accuracy and versatility of the IIL real-time emulation system, the system-level (1 microsecond time-step) and device-level (50 nanosecond time-step) results are compared in three domains: the referencer system (C code simulation program in NVIDIA Jetson and offline SaberRD datasets), offline inferencer emulation on Xilinx VCU118 board, and online refined inferencer emulation on Xilinx VCU118 board.