Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Software
Data sources: ZENODO
addClaim

streamrelay: WebSocket Relay for Real-Time Streaming from Batch HPC Executors

Authors: Nassar, Anas;

streamrelay: WebSocket Relay for Real-Time Streaming from Batch HPC Executors

Abstract

streamrelay is a domain-agnostic Python library for streaming incremental output from batch HPC jobs to remote clients in real time. It implements a dual-channel relay pattern: a producer (running inside the HPC job) connects outbound to a relay server and forwards output token by token; a consumer (the client) connects outbound to the same channel and receives output as it is generated. Neither side accepts inbound connections, making the pattern firewall-transparent. While demonstrated for LLM token streaming via vLLM, the library is not LLM-specific and generalizes to any HPC job that produces incremental output, such as simulations, data pipelines, or scientific workflows. PyPI: https://pypi.org/project/streamrelay/ Source code: https://github.com/uicacer/streamrelay

Powered by OpenAIRE graph
Found an issue? Give us feedback