Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Software
Data sources: ZENODO
addClaim

Harbor: A framework for evaluating and optimizing agents and models in container environments

Authors: Harbor Framework Team;

Harbor: A framework for evaluating and optimizing agents and models in container environments

Abstract

What's Changed Fix FX installation and background persistence by @DarlingtonDeveloper in https://github.com/harbor-framework/harbor/pull/2829 Migrate agents to structured capability declarations by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2834 fix(rewardkit): trajectory criteria default to /logs/agent/trajectory.json by @no-hup in https://github.com/harbor-framework/harbor/pull/2840 fix(islo): slug ephemeral gateway profile names by @tomerezer in https://github.com/harbor-framework/harbor/pull/2844 fix(vercel): auto-load .env.local so onboarding auth flow doesn't loop by @erulkey in https://github.com/harbor-framework/harbor/pull/2845 fix(qwen-code): preserve reasoning and subagent trajectories by @DragonnZhang in https://github.com/harbor-framework/harbor/pull/2364 Fix/hyperbrowser remote image builds by @Dingway98 in https://github.com/harbor-framework/harbor/pull/2833 Docs trajectory loading by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2859 Docs handoff by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2860 docs regrade by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2864 Add swe interact example task by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2865 Google Sans Code logo by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2861 Docs simulated user by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2866 docs: remote rollouts guide for hosted Harbor by @scvance in https://github.com/harbor-framework/harbor/pull/2867 refactor(hub)!: rename the secret's provider field to --label by @scvance in https://github.com/harbor-framework/harbor/pull/2862 docs: add usage stats by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2884 docs: job results viewer by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2889 fix: extend mini-swe LiteLLM timeout to 3600s by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2886 fix(hosted): make credential selection explicit for hosted launches by @scvance in https://github.com/harbor-framework/harbor/pull/2857 feat: add Podman as a ContainerRuntime behind DockerEnvironment by @xichen1997 in https://github.com/harbor-framework/harbor/pull/2872 docs: results handoff by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2890 docs: add custom agents guide by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2892 docs: add pre-integrated agents by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2891 Docs acp agents by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2894 docs: use Geist and Google Sans Code on the Mintlify site by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2828 fix doc by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2897 docs: add Harbor tasks overview by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2898 docs: add ATIF trajectory by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2899 docs: "uv run harbor" -> "harbor" by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2901 docs: -p -> -t by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2900 fix(viewer): prefer canonical reward in job results by @ayushnangia in https://github.com/harbor-framework/harbor/pull/2877 feat(tensorlake sandbox): dynamic network policy + docker-in-docker support by @cooleel in https://github.com/harbor-framework/harbor/pull/2817 Simplify nested RewardKit groups by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/2836 Use kebab-case RewardKit aggregations by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/2906 fix: classify new Anthropic refusals pattern by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2922 fix(env): do not classify TOKENS as secrets by @scvance in https://github.com/harbor-framework/harbor/pull/2925 Fix Windows cross-drive artifact paths by @MarcoRossignoli in https://github.com/harbor-framework/harbor/pull/2878 feat(atif): support audio content parts (ATIF-v1.8) by @miguelrc-scale in https://github.com/harbor-framework/harbor/pull/2605 feat(computer-1): support Gemini 3.7 Flash computer use by @erikqu in https://github.com/harbor-framework/harbor/pull/2936 docs: add environment variable by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2926 docs: add artifact collection by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2937 docs: add custom sandbox by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2935 docs: add CLI, config, Python to preinstalled agents by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2934 fix(verifier): reject non-finite (NaN/Infinity) reward values at ingestion by @no-hup in https://github.com/harbor-framework/harbor/pull/2888 Add fx dev-channel agent by @fazxes in https://github.com/harbor-framework/harbor/pull/2932 Add configurable session ID headers by @shariqm-modal in https://github.com/harbor-framework/harbor/pull/2723 Detect likely Modal sandbox OOM exits by @scvance in https://github.com/harbor-framework/harbor/pull/2955 docs: add existing plugins guide by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2957 docs: pre-integrated sandboxes by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2933 Validate wheel extras before publish by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2960 docs: add release policy by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2971 docs: explain separate verifier environment variables by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2970 Improve mintlify navigation by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2969 docs: add custom plugins by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2959 docs: add versioning by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2966 fix(rewardkit): don't double-register a reweighted zero-parameter criterion by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/2945 Fix ACP bootstrap on older task images by @DarlingtonDeveloper in https://github.com/harbor-framework/harbor/pull/2973 fix(skills): Handle multiple git remote urls from git remote-ls in skill resuolution by @cdxker in https://github.com/harbor-framework/harbor/pull/2958 fix: remove tensorlake from cloud extra by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2980 docs: reduce ATIF wording by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2902 Default harbor auth login to the manual flow in SSH sessions by @devin-ai-integration[bot] in https://github.com/harbor-framework/harbor/pull/2981 fix(openhands-sdk): forward reasoning effort to LiteLLM by @sravell in https://github.com/harbor-framework/harbor/pull/2927 fix: restore tensorlake in cloud extra by @cooleel in https://github.com/harbor-framework/harbor/pull/2991 docs: illustrate network policy phases by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2990 docs: document agent capabilities by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2989 docs: clarify changelog in CONTRIBUTING.md by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2988 Docs curated changelog by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2982 docs: update stable release policy by @kobe0938 in https://github.com/harbor-framework/harbor/pull/2984 fix(docker): stream stdout without a line-length limit by @somu84 in https://github.com/harbor-framework/harbor/pull/2985 feat(acp): support task-provided local agents by @andy-zhou in https://github.com/harbor-framework/harbor/pull/2992 opencode: opt out of OAuth for remote MCP servers by @psbang in https://github.com/harbor-framework/harbor/pull/2918 antigravity-sdk: forward GOOGLE_GEMINI_BASE_URL to the runner by @psbang in https://github.com/harbor-framework/harbor/pull/2917 Bound the Antigravity model discovery command timeout by @alexgshaw in https://github.com/harbor-framework/harbor/pull/2999 Pin mini-swe-agent installer Python by @RyanMarten in https://github.com/harbor-framework/harbor/pull/3008 Add fx agent judge to RewardKit by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/2848 fix(rewardkit): decide tests/ layout from registered criteria by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/2944 feat(rewardkit): signed weighted aggregation and validated judge TOMLs by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/2879 docs: fill asp sandbox page with rfc 0003 content by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3024 feat: add kata environment for KVM microVM isolation by @xichen1997 in https://github.com/harbor-framework/harbor/pull/2998 small fix on asp rfc. by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3026 docs: drop stale pinned version from README citation block by @StevenDillmann in https://github.com/harbor-framework/harbor/pull/3027 Run HF sandbox commands with Bash by @evalstate in https://github.com/harbor-framework/harbor/pull/3016 feat(claude-code): add thinking_display kwarg (--thinking-display) by @hrdkbhatnagar in https://github.com/harbor-framework/harbor/pull/3030 Support composing job config files by @walkerhughes in https://github.com/harbor-framework/harbor/pull/3022 docs: add Hosted Harbor Mintlify section by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3051 Add Muse Code installed agent integration by @think-step-by-step in https://github.com/harbor-framework/harbor/pull/3043 Improve agent error classification by @scvance in https://github.com/harbor-framework/harbor/pull/3055 fix(acp): make managed Python readable by agent by @andy-zhou in https://github.com/harbor-framework/harbor/pull/3015 Declare and validate built-in agent options by @alexgshaw in https://github.com/harbor-framework/harbor/pull/3049 docs: agent option discovery by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3062 docs: add Muse Code agent references by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3057 ci: run Linux tests on Ubuntu 26.04 by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3075 modal: simplify and default adoption of v2 system by @thundergolfer in https://github.com/harbor-framework/harbor/pull/3077 feat(hermes): support multi-step resume by @ayushnangia in https://github.com/harbor-framework/harbor/pull/3045 fix(environments): give the transfer archive pack a realistic timeout and name it on silent failures by @hrdkbhatnagar in https://github.com/harbor-framework/harbor/pull/3046 Add Hub job rename and ownership transfer commands by @scvance in https://github.com/harbor-framework/harbor/pull/3058 fix: update Codex maximum reasoning effort by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3072 fix: remove unsupported Claude Code effort by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3073 refactor(viewer): drive agent options from schema by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3069 fix(codex): accept none reasoning effort for luna by @alexgshaw in https://github.com/harbor-framework/harbor/pull/3087 feat(swe-agent): expose completion_kwargs by @amy-g-chen in https://github.com/harbor-framework/harbor/pull/3048 Reuse cached content-addressed task packages without Hub resolution by @alexgshaw in https://github.com/harbor-framework/harbor/pull/3101 Add Harbor Hub agent catalog discovery by @alexgshaw in https://github.com/harbor-framework/harbor/pull/3061 Refine harbor run help text by @alexgshaw in https://github.com/harbor-framework/harbor/pull/3109 Add job config validation and metadata-only dry runs by @alexgshaw in https://github.com/harbor-framework/harbor/pull/3110 Expand hosted retry exclusions by @scvance in https://github.com/harbor-framework/harbor/pull/3081 Add Runta cloud sandbox environment by @xichen1997 in https://github.com/harbor-framework/harbor/pull/2738 Add static outbound IP to shared hosted configuration by @scvance in https://github.com/harbor-framework/harbor/pull/3137 docs: clarify RewardKit isolation requirements by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/3063 fix(rewardkit): send trajectory media to the judge model by @benediktstroebl in https://github.com/harbor-framework/harbor/pull/3047 Adjust task copying by @scvance in https://github.com/harbor-framework/harbor/pull/3149 Sandbox prompt templates by @scvance in https://github.com/harbor-framework/harbor/pull/3150 Daytona: support H200, RTX 4090, RTX 5090 and MI355X by @sruan2 in https://github.com/harbor-framework/harbor/pull/3170 feat(strands): add installed agent integration for the Strands Agents SDK by @jjbuck in https://github.com/harbor-framework/harbor/pull/3089 Improve task validation and file handling by @scvance in https://github.com/harbor-framework/harbor/pull/3148 fix(gke): tolerate binary websocket frames in exec output by @astronautas in https://github.com/harbor-framework/harbor/pull/3079 Support scoped resumable upload tokens by @scvance in https://github.com/harbor-framework/harbor/pull/2928 feat(antigravity-cli): report token usage and cost by @RileyHYZ in https://github.com/harbor-framework/harbor/pull/2885 feat: record per-model token usage in agent_result by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3178 feat(claude-code): keep per-step cost estimates for per-model usage by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3179 Fix ACP installer checkout ownership for non-root users by @scvance in https://github.com/harbor-framework/harbor/pull/3181 Fix ACP installer cache permissions by @scvance in https://github.com/harbor-framework/harbor/pull/3183 Regrade completed multi-step trials by @ayushnangia in https://github.com/harbor-framework/harbor/pull/2907 style: indent ACP dependency installation script by @kobe0938 in https://github.com/harbor-framework/harbor/pull/3185 fix(antigravity-sdk): take token totals from the SDK session aggregate by @vsiva in https://github.com/harbor-framework/harbor/pull/3118 New Contributors @DarlingtonDeveloper made their first contribution in https://github.com/harbor-framework/harbor/pull/2829 @no-hup made their first contribution in https://github.com/harbor-framework/harbor/pull/2840 @tomerezer made their first contribution in https://github.com/harbor-framework/harbor/pull/2844 @erulkey made their first contribution in https://github.com/harbor-framework/harbor/pull/2845 @DragonnZhang made their first contribution in https://github.com/harbor-framework/harbor/pull/2364 @ayushnangia made their first contribution in https://github.com/harbor-framework/harbor/pull/2877 @cdxker made their first contribution in https://github.com/harbor-framework/harbor/pull/2958 @devin-ai-integration[bot] made their first contribution in https://github.com/harbor-framework/harbor/pull/2981 @sravell made their first contribution in https://github.com/harbor-framework/harbor/pull/2927 @somu84 made their first contribution in https://github.com/harbor-framework/harbor/pull/2985 @andy-zhou made their first contribution in https://github.com/harbor-framework/harbor/pull/2992 @evalstate made their first contribution in https://github.com/harbor-framework/harbor/pull/3016 @hrdkbhatnagar made their first contribution in https://github.com/harbor-framework/harbor/pull/3030 @thundergolfer made their first contribution in https://github.com/harbor-framework/harbor/pull/3077 @amy-g-chen made their first contribution in https://github.com/harbor-framework/harbor/pull/3048 @sruan2 made their first contribution in https://github.com/harbor-framework/harbor/pull/3170 @jjbuck made their first contribution in https://github.com/harbor-framework/harbor/pull/3089 @astronautas made their first contribution in https://github.com/harbor-framework/harbor/pull/3079 @RileyHYZ made their first contribution in https://github.com/harbor-framework/harbor/pull/2885 @vsiva made their first contribution in https://github.com/harbor-framework/harbor/pull/3118 Full Changelog: https://github.com/harbor-framework/harbor/compare/v0.22.0...v0.23.0

Powered by OpenAIRE graph
Found an issue? Give us feedback