ci(changesets): version packages

[voltagent-bot]

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.

Releases

@voltagent/[email protected]

Minor Changes

• #1055 21891b4 Thanks @omeraplak! - feat: add tool-aware live-eval payloads and a deterministic tool-call accuracy scorer

  What's New

  • @voltagent/core
    • Live eval payload now includes messages, toolCalls, and toolResults.
    • If toolCalls/toolResults are not explicitly provided, they are derived from the normalized message/step chain.
    • New exported eval types: AgentEvalToolCall and AgentEvalToolResult.
  • @voltagent/scorers
    • Added prebuilt createToolCallAccuracyScorerCode for deterministic tool evaluation.
    • Supports both single-tool checks (expectedTool) and ordered tool-chain checks (expectedToolOrder).
    • Supports strict and lenient matching modes.

  Code Examples

  Built-in tool-call scorer:

  import { createToolCallAccuracyScorerCode } from "@voltagent/scorers";
  
  const toolOrderScorer = createToolCallAccuracyScorerCode({
    expectedToolOrder: ["searchProducts", "checkInventory"],
    strictMode: false,
  });

  Custom scorer using toolCalls + toolResults:

  import { buildScorer } from "@voltagent/core";
  
  interface ToolEvalPayload extends Record<string, unknown> {
    toolCalls?: Array<{ toolName?: string }>;
    toolResults?: Array<{ isError?: boolean; error?: unknown }>;
  }
  
  const toolExecutionHealthScorer = buildScorer<ToolEvalPayload, Record<string, unknown>>({
    id: "tool-execution-health",
    label: "Tool Execution Health",
  })
    .score(({ payload }) => {
      const toolCalls = payload.toolCalls ?? [];
      const toolResults = payload.toolResults ?? [];
  
      const failedResults = toolResults.filter(
        (result) => result.isError === true || Boolean(result.error)
      );
      const completionRatio =
        toolCalls.length === 0 ? 1 : Math.min(toolResults.length / toolCalls.length, 1);
  
      return {
        score: Math.max(0, completionRatio - failedResults.length * 0.25),
        metadata: {
          toolCallCount: toolCalls.length,
          toolResultCount: toolResults.length,
          failedResultCount: failedResults.length,
        },
      };
    })
    .build();

• #1054 3556385 Thanks @omeraplak! - feat: add onToolError hook for customizing tool error payloads before serialization

  Example:

  import { Agent, createHooks } from "@voltagent/core";
  
  const agent = new Agent({
    name: "Assistant",
    instructions: "Use tools when needed.",
    model: "openai/gpt-4o-mini",
    hooks: createHooks({
      onToolError: async ({ originalError, error }) => {
        const maybeAxios = (originalError as any).isAxiosError === true;
        if (!maybeAxios) {
          return;
        }
  
        return {
          output: {
            error: true,
            name: error.name,
            message: originalError.message,
            code: (originalError as any).code,
            status: (originalError as any).response?.status,
          },
        };
      },
    }),
  });

@voltagent/[email protected]

Minor Changes

• #1055 21891b4 Thanks @omeraplak! - feat: add tool-aware live-eval payloads and a deterministic tool-call accuracy scorer

  What's New

  • @voltagent/core
    • Live eval payload now includes messages, toolCalls, and toolResults.
    • If toolCalls/toolResults are not explicitly provided, they are derived from the normalized message/step chain.
    • New exported eval types: AgentEvalToolCall and AgentEvalToolResult.
  • @voltagent/scorers
    • Added prebuilt createToolCallAccuracyScorerCode for deterministic tool evaluation.
    • Supports both single-tool checks (expectedTool) and ordered tool-chain checks (expectedToolOrder).
    • Supports strict and lenient matching modes.

  Code Examples

  Built-in tool-call scorer:

  import { createToolCallAccuracyScorerCode } from "@voltagent/scorers";
  
  const toolOrderScorer = createToolCallAccuracyScorerCode({
    expectedToolOrder: ["searchProducts", "checkInventory"],
    strictMode: false,
  });

  Custom scorer using toolCalls + toolResults:

  import { buildScorer } from "@voltagent/core";
  
  interface ToolEvalPayload extends Record<string, unknown> {
    toolCalls?: Array<{ toolName?: string }>;
    toolResults?: Array<{ isError?: boolean; error?: unknown }>;
  }
  
  const toolExecutionHealthScorer = buildScorer<ToolEvalPayload, Record<string, unknown>>({
    id: "tool-execution-health",
    label: "Tool Execution Health",
  })
    .score(({ payload }) => {
      const toolCalls = payload.toolCalls ?? [];
      const toolResults = payload.toolResults ?? [];
  
      const failedResults = toolResults.filter(
        (result) => result.isError === true || Boolean(result.error)
      );
      const completionRatio =
        toolCalls.length === 0 ? 1 : Math.min(toolResults.length / toolCalls.length, 1);
  
      return {
        score: Math.max(0, completionRatio - failedResults.length * 0.25),
        metadata: {
          toolCallCount: toolCalls.length,
          toolResultCount: toolResults.length,
          failedResultCount: failedResults.length,
        },
      };
    })
    .build();

Patch Changes

• Updated dependencies [21891b4, 3556385]:
  • @voltagent/[email protected]

Summary by CodeRabbit

• New Features

  • Added onToolError hook to customize tool error payloads before serialization.
  • Added tool-aware live-eval payloads and deterministic tool-call scorer for enhanced evaluation capabilities.

• Documentation

  • Updated changelogs with feature details and usage examples for version 2.4.0 and 2.1.0 releases.

[cloudflare-workers-and-pages]

Deploying voltagent with    Cloudflare Pages

Latest commit:	5bc2cf0
Status:	✅  Deploy successful!
Preview URL:	https://85761186.voltagent.pages.dev
Branch Preview URL:	https://changeset-release-main.voltagent.pages.dev

View logs

[coderabbitai]

📝 Walkthrough

Walkthrough

Removed a release changeset; bumped packages/core to 2.4.0 and documented a new onToolError hook in core changelog; bumped @voltagent/core in many example package.json files; bumped packages/scorers to 2.1.0 and updated its changelog and core dependency.

Changes

Cohort / File(s)	Summary
Release Cleanup \.changeset/good-bottles-wave.md	Removed changeset entry used for the release.
Core Package packages/core/package.json, packages/core/CHANGELOG.md	Version bumped to 2.4.0; changelog adds onToolError hook and usage example documenting error payload transformation.
Scorers Package packages/scorers/package.json, packages/scorers/CHANGELOG.md	Version bumped to 2.1.0; changelog updated and dependency on @voltagent/core set to ^2.4.0.
Example Projects examples/*/package.json	Updated @voltagent/core dependency from ^2.3.8 → ^2.4.0 across the example projects (many package.json files changed).

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant Dev as Developer
  participant Agent as Agent Runtime
  participant Tool as Tool (call)
  participant Hook as onToolError Hook
  participant Serializer as Error Serializer

  Dev->>Agent: triggers tool execution
  Agent->>Tool: call tool
  Tool-->>Agent: throws error
  Agent->>Hook: invoke onToolError(error, context)
  Hook-->>Agent: returns transformed payload
  Agent->>Serializer: serialize transformed payload
  Serializer-->>Dev: delivered tool error payload

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• ci(changesets): version packages #982 — touches agent/tool hooks and is related to tool hook behavior changes.

Poem

🐰 A tiny hop for changelog cheer,
I nibbled bugs and bumped a year.
Hooks for errors, examples all bright,
Versions climb and docs take flight.
Thump-thump — release carrots in sight! 🥕✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	The PR description is detailed and complete, describing the releases, features, code examples, and dependencies. However, it does not follow the repository's PR description template (missing sections like PR Checklist, Bugs/Features, What is the current behavior, etc.).	This is an automated Changesets release PR with a standard format. Consider whether the template applies to auto-generated release PRs, or if the detailed release notes format is acceptable for this use case.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'ci(changesets): version packages' clearly and accurately summarizes the main change—this is a Changesets-driven automated release PR that bumps package versions.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch changeset-release/main

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[cubic-dev-ai]

No issues found across 82 files

Note: This PR contains a large number of files. cubic only reviews up to 75 files per PR, so some files may not have been reviewed.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

packages/scorers/package.json (1)

33-34: ⚠️ Potential issue | 🟡 Minor

Peer dependency for @voltagent/core may be too broad.

The direct dependency requires ^2.4.0 (line 7), indicating scorers 2.1.0 relies on APIs introduced in core 2.4.0 (e.g., AgentEvalToolCall, AgentEvalToolResult). However, the peer dependency still allows ^2.0.0, which could let consumers install an older core version that lacks these APIs, potentially causing runtime errors when npm dedupes.

Consider tightening the peer dependency to ^2.4.0 to match.