Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
addClaim

experimaestro/datamaestro: v1.14.0 — pluggable hf_resolver helpers + env-driven local mirror

Authors: Benjamin Piwowarski;

experimaestro/datamaestro: v1.14.0 — pluggable hf_resolver helpers + env-driven local mirror

Abstract

Highlights New datamaestro.helpers module: pluggable helpers contributed by third-party packages through the datamaestro.helpers entry-point group. HFResolver Protocol — redirect HF Hub repos to a local directory (e.g. an HPC cluster's shared model/dataset mirror). Built-in _EnvHFResolver driven by colon-separated env vars: DATAMAESTRO_HF_MODELS_CACHE (model repos: // with config.json) DATAMAESTRO_HF_DATASETS_CACHE (dataset repos: //) Always registered — no plugin required for the env-driven flow. HFDownloader and HFSnapshotDownloader consult get_helpers("hf_resolver") before reaching the network. No symlinks, no copies — the mirror directory is exposed as the resource path directly. Use case On HPC clusters with a pre-mirrored HF cache (e.g. Jean-Zay's $DSDIR/HuggingFace_Models / $DSDIR/HuggingFace): export DATAMAESTRO_HF_MODELS_CACHE=$DSDIR/HuggingFace_Models export DATAMAESTRO_HF_DATASETS_CACHE=$DSDIR/HuggingFace — HF lookups are served from the shared mirror with zero user-quota cost. Distinctive INFO log lines fire on every hit. Tests Adds 13 tests covering the env resolver (single & colon-separated roots, missing-config-json case, dataset lookup, Protocol compliance), plugin layering via entry points, and short-circuit behaviour of both downloaders.

  • BIP!
    Impact byBIP!
    selected citations
    These citations are derived from selected sources.
    This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    0
    popularity
    This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
    Average
    influence
    This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
    Average
    impulse
    This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
    Average
Powered by OpenAIRE graph
Found an issue? Give us feedback
selected citations
These citations are derived from selected sources.
This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Citations provided by BIP!
popularity
This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network.
BIP!Popularity provided by BIP!
influence
This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically).
BIP!Influence provided by BIP!
impulse
This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network.
BIP!Impulse provided by BIP!
0
Average
Average
Average