Archives preserve content and support various types of analysis, study, and use. Archiving the Web and social media (e.g., Twitter) content involves use of representations like Web ARChive (WARC) and JavaScript Object Notation (JSON). Parquet is a newer representation that yields better performance with large collections for many data processing tasks. In support of archive-related research at Virginia Tech in University Libraries and the Digital Library Research Laboratory (DLRL), this project involved the development of software (tested in Linux and Windows, as well as on local and container cluster environments), and its packaging in containers, for converting WARC to Parquet as well as JSON to Parquet. Images of the resulting containers are available to support the research of others.

ParquetContainersReport.zip = ZIP of the LaTeX files from Overleaf for the report. ParquetContainersReport.pdf = PDF version of the report. ParquetContainersPresentation.pdf = PDF version of the final presentation. ParquetContainersPresentation.pptx = Final presentation version for PowerPoint.