Sound transmission and scattering properties in higher-order modes across the tee-junction of a rectangular duct used in ventilation and air-conditioning system were investigated numerically and experimentally. High-sound transmission of the fundamental mode and higher-order modes across the main duct is observed at eigen-frequencies of the main duct. The resonance of branch modes is suppressed by the weak modal coupling of the branch-modes and the traveling wave in the main duct at or very close to the eigen-frequencies of the sidebranch, which results in high-sound transmission of the fundamental mode and higher-order modes across the main duct and excitation of the branch modes at higher frequencies. Increases in sound scattering into higher-order mode are found when the non-planar or longitudinal branch-mode excited. In the case of co-excitation of the longitudinal branch-mode and non-planar branch-modes, a broader band-stop action in sound transmission has been observed. The results of numerical simulations were verified by experiments. A formulation of a transmission matrix based on the transfer function and a two-microphone method was shown. [The work described in this paper was supported by a grant from the Hong Kong Polytechnic University (Project A/C Code: G-U362).]