Abstract Objectives Observational data have been actively used to estimate treatment effect, driven by the growing availability of electronic health records (EHRs). However, EHRs typically consist of longitudinal records, often introducing time-dependent confounding that hinder the unbiased estimation of treatment effect. Inverse probability of treatment weighting (IPTW) is a widely used propensity score method since it provides unbiased treatment effect estimation and its derivation is straightforward. In this study, we aim to utilize IPTW to estimate treatment effect in the presence of time-dependent confounding using claims records. Materials and Methods Previous studies have utilized propensity score methods with features derived from claims records through feature processing, which generally requires domain knowledge and additional resources to extract information to accurately estimate propensity scores. Deep learning, particularly using deep sequence models such as recurrent neural networks and Transformer, has demonstrated good performance in modeling EHRs for various downstream tasks. We propose that these deep sequence models can provide accurate IPTW estimation of treatment effect by directly estimating the propensity scores from claims records without the need for feature processing. Results Comprehensive evaluations on synthetic and semi-synthetic datasets demonstrate that IPTW treatment effect estimation using deep sequence models consistently outperforms baseline approaches, including logistic regression and multilayer perceptrons, combined with feature processing. Discussion Our findings demonstrate that deep sequence models consistently outperform traditional approaches in estimating treatment effects, particularly under time-dependent confounding. Moreover, Transformer-based models offer interpretability by assigning higher attention weights to relevant confounders, even when prior domain knowledge is limited. Conclusion Deep sequence models enable accurate treatment effect estimation through IPTW without the need for feature processing.