Reliable anomaly detection in concrete infrastructure is often hindered by extreme label scarcity and significant data shifts across diverse inspection sites. To address these challenges, we propose an autonomous, site-specific diagnostic framework that operates without site-specific training or expert-curated labels. Our unsupervised approach integrates the Wavelet Scattering Transform (WST) and Isolation Forest (iForest). We demonstrate that second-order scattering coefficients are essential for capturing high-frequency amplitude modulations, recovering transient resonance signatures that exceed human auditory perception. By employing an isolation-based mechanism, the framework identifies anomalies as isolated points in the high-dimensional feature space, ensuring operational stability without delicate hyperparameter tuning or large training sets. Experimental results on reinforced concrete specimens yield a mean ROC-AUC of 0.8852, consistently outperforming CWT-Autoencoder baselines. Furthermore, SHAP-based interpretability analysis confirms that our model's decisions are grounded in structural physics, successfully identifying deep internal cracks that were imperceptible to human inspectors.This preprint was submitted to ICONIP 2026 on April 29, 2026 and is currently under review.