Graph theory channel modeling is an efficient approach to simulate multipath radio propagation including the reverberation effect of electromagnetic waves. In this contribution, without modifying the modeling framework, we proposed a semi-deterministic channel modeling approach by associating the scatterers with realistic environment objects, and by calculating the coefficients of the propagation paths on the base of a physically sound and proven diffuse scattering theory. The diffuse multipath components are then combined with the specular components (SCs) simulated by ray-tracing (RT) to obtain a complete channel representation. The proposed method is evaluated in two reference scenarios at 3.8 and 60 GHz, respectively, by comparing the simulated channel characteristics with channel measurement data. Results show that the proposed method can accurately predict the channel characteristics in both the delay domain and the angular domain. The proposed approach is appropriate to model multipath propagation in confined indoor or dense-urban environment at millimeter (mm)-wave frequencies and above, where reverberation and rough-surface scattering can be important phenomena.