
PLEASE NOTE: Following the conference, it is important to note that the approach described in this abstract was influenced by time and infrastructure constraints. While effective at the scale of ARGA’s datasets in this specific context, embedding large data transfers within a GraphQL response is not generally recommended. In most cases, a more conventional approach - such as generating an export job, writing the result to object storage (e.g., S3 or GCS), and providing download via HTTP - will be more scalable, maintainable, and operationally robust. The Australian Reference Genome Atlas (ARGA), an advanced indexing platform designed to aggregate, filter, and provide access to complex genomics datasets, saw a need to support performant downloads of large datasets without introducing bespoke infrastructure. The primary objective was to enable users to efficiently retrieve large-scale CSV data through the existing GraphQL API. We also sought to avoid standalone download endpoints that would introduce maintenance overhead and architectural complexity. Our solution was two-pronged, leveraging backend and client-side implementations: 1. We leveraged the performance of ARGA s Rust-based backend and the flexibility of GraphQL. The backend retrieves the necessary records directly from ARGA s PostgreSQL database and transforms them into a CSV-formatted string. This CSV content is then compressed using Brotli, a modern compression algorithm known for its high compression ratio and decompression speed. The resulting binary data is encoded into a base64 string to ensure compatibility with the GraphQL transport layer and transmitted to the client through a standard GraphQL response. 2. On the client side, the base64 string is decoded, and WebAssembly (WASM) modules are used to decompress the Brotli-encoded data directly within the browser. This approach leverages WASM s efficiency and sandboxed execution to minimise latency and browser resource consumption. The solution achieved a significant reduction in payload size from 56.1 MB uncompressed down to 12.1 MB compressed for a sample dataset of 265,459 records. In this presentation, we demonstrate the novel utilisation of compression techniques to enhance the performance and efficiency of web-based applications.
Tools, Research Software
Tools, Research Software
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
