SciLaD is a novel, large-scale dataset of scientific language constructed entirely using open-source frameworksand publicly available data sources. It comprises a curated English split containing over 10 million scientificpublications and a multilingual, unfiltered TEI XML split including more than 35 million publications. We also publishthe extensible pipeline for generating SciLaD. The dataset construction and processing workflow demonstrateshow open-source tools can enable large-scale, scientific data curation while maintaining high data quality. Finally,we pre-train a RoBERTa model on our dataset and evaluate it across a comprehensive set of benchmarks,achieving performance comparable to other scientific language models of similar size, validating the quality andutility of SciLaD. We publish the dataset and evaluation pipeline to promote reproducibility, transparency, andfurther research in natural scientific language processing and understanding, including scholarly document processing.