Powered by OpenAIRE graph
Found an issue? Give us feedback
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/ ZENODOarrow_drop_down
image/svg+xml art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos Open Access logo, converted into svg, designed by PLoS. This version with transparent background. http://commons.wikimedia.org/wiki/File:Open_Access_logo_PLoS_white.svg art designer at PLoS, modified by Wikipedia users Nina, Beao, JakobVoss, and AnonMoos http://www.plos.org/
ZENODO
Software
Data sources: ZENODO
addClaim

truera/trulens: TruLens 2.14.0

Authors: Josh Reini; Josh Reini; David Kurokawa; Piotr Mardziel; rshih32; Daniel Huang; corey hu; +21 Authors

truera/trulens: TruLens 2.14.0

Abstract

TruLens 2.14.0 Release Notes Headline Features Coding Agent Instrumentation Instrument IDE hooks and coding agent workflows — including Claude Code and Cursor — with TruLens tracing. Capture the full execution trace of agentic coding sessions for evaluation and observability. (#2732) by @joshreini1 Production Trace Curation into Evaluation Datasets Curate production traces into persisted evaluation datasets. Select and save real-world traces directly from your running application, building ground-truth datasets for offline evaluation without manual data collection. (#2722) by @Payal2000 Connector-Backed Prompt Management with Version Evaluation Store, version, and evaluate prompt templates through the connector interface. Track prompt changes over time and evaluate the impact of prompt edits against your existing datasets and feedback functions. (#2717) by @dylanpulver OTLP-Native Export and OTel Metrics Signal Export traces and metrics directly via the OpenTelemetry Protocol, enabling integration with any OTLP-compatible backend. (#2681) by @RyanFlowerYes Additional Features Streaming token tracking with TTFT and throughput metrics — OTEL instrumentation now captures streaming token counts, time-to-first-token, and throughput for LLM calls. (#2708) by @Abelo9996 Ollama provider — New trulens-providers-ollama package for running feedback functions against local Ollama models. (#2688) by @connectsudhindra-gif Conversation metric customization — additional_instructions and chain-of-thought reasons support for conversation-scoped metrics. (#2710) by @dylanpulver Dashboard Improvements Leaderboard record counts — Leaderboard metrics now show how many records each metric actually scored. (#2698) by @CTWalk TTFT in the Records page — Time-to-first-token is now visible in the latency view on the Records page. (#2696) by @connectsudhindra-gif Conversation thread column labels — Conversation-scoped thread columns are now labeled "latest" instead of "avg" to reflect their actual semantics. (#2750) by @Abelo9996 Topic Adherence score consistency — Topic Adherence scores now match between the results table and conversation metrics view. (#2733) by @Abelo9996 Bug Fixes LLM judge rating extraction — Fixed a bug where the judge's stated scale description was incorrectly parsed as its numeric rating. (#2726) by @arthi-arumugam-git ndcg_at_k shape crash and cohens_kappa label mutation — ndcg_at_k no longer crashes on mismatched shapes; cohens_kappa no longer mutates its input labels. (#2709) by @ClaireXi99 Ground-truth row ordering — Benchmark scoring now processes ground-truth rows in row order, not index-label order. (#2737) by @arthi-arumugam-git IR metric undefined results — IR metrics now report nan instead of 0.0 when undefined (e.g., empty relevance lists). (#2738) by @arthi-arumugam-git Duplicate chunk handling in IR metrics — recall_at_k and ndcg_at_k are now correctly bounded when retrieved chunks contain duplicates. (#2747) by @Abelo9996 Temperature argument for conversation metrics — temperature stays positional on conversation metric calls, fixing a regression. (#2728) by @BetterAndBetterII Dataset and ground-truth read round-trips — Fixed serialization round-trip errors when reading datasets and ground-truth entries from the database. (#2735) by @Abelo9996 Feedback definition re-insert — Feedback definition updates are now persisted correctly on re-insert. (#2743) by @Abelo9996 compute_metrics message formatting — Status messages now build from metric names, not raw objects. (#2745) by @Abelo9996 Anthropic cost tracking — Fixed a crash in the Anthropic provider when tracking costs via core_endpoint.Cost. (#2756) by @Abelo9996 methods_to_instrument existence check — App wrappers now use Lens.get for safer instrumentation target checks. (#2754) by @Abelo9996 OpenAI moderation_harassment_threatening — This moderation method now reads the correct category from the API response. (#2752) by @Abelo9996 OTEL tracing default cleanup — Removed stale enablement references; TruLens now warns when OTEL tracing is explicitly disabled. (#2724) by @sfc-gh-jreini Ruff version alignment — All three ruff pins are now aligned on 0.16.3. (#2720) by @dylanpulver Conda recipe updates — Conda recipes updated for TruLens 2.13.0. (#2713) by @sfc-gh-jreini Missing docstring fix — Added missing docstring on public function pytest_collection_modi. (#2706) by @nickhac New example Trace debugging cookbook — New notebook walking through common trace debugging workflows. (#2716) by @dylanpulver New Contributors @dylanpulver made their first contribution in #2710 @ClaireXi99 made their first contribution in #2709 @Abelo9996 made their first contribution in #2708 @nickhac made their first contribution in #2706 @CTWalk made their first contribution in #2698 @arthi-arumugam-git made their first contribution in #2726 @connectsudhindra-gif made their first contribution in #2688 @BetterAndBetterII made their first contribution in #2728

Powered by OpenAIRE graph
Found an issue? Give us feedback