Introduction

callcheck validates AI tool call responses against a schema and auto repairs the ones that fail. Built on Zod, works with any AI provider, free through Vercel AI Gateway.

Install

npm install -g callcheck

Schema file

Describe the expected shape of a tool call response in a JSON file. Supports string, number, boolean, email, plus enums and number constraints.

{
  "userId": "string",
  "email": "email",
  "plan": { "type": "enum", "values": ["free", "pro", "enterprise"] },
  "status": { "type": "enum", "values": ["active", "suspended", "pending"] }
}

Run a check

callcheck check --schema schema.json --response response.json

A valid response returns:

passed response matches schema

An invalid response returns:

✘ failed response does not match schema

- email: invalid email format

- plan: invalid enum value

Auto repair

Add --repair to send a failing response to a model for correction. Requires a free AI Gateway key, no separate model subscription needed.

callcheck check --schema schema.json --response response.json --repair

Set your key once per terminal session:

$env:AI_GATEWAY_API_KEY = "your-key-here"

Wrap a tool once

In code, wrap any AI SDK style tool definition. Every call through it gets validated and repaired automatically after that, no extra code downstream.

import { validated, gatewayRetry } from "callcheck";

const queryUser = validated(queryUserTool, {
  repair: true,
  retry: gatewayRetry(process.env.AI_GATEWAY_API_KEY),
});

Track failures

Every repair run gets logged locally to .callcheck/runs.json. See which fields and which models fail most.

callcheck report

Use it in CI

callcheck exits with a non zero code on failure, so it fits directly into a pipeline step.

- name: Validate tool response
  run: callcheck check --schema schema.json --response response.json