agentutility(7)$ man eval-case-generate
/eval-case-generate
PRICE / CALL
$0.02
USDC · base mainnet · scheme: exact
METHOD
POST
CLUSTER
agentopsCATEGORY
uncategorized
STATUS
● live
NAME
eval-case-generate — eval case generator for an agent or llm task: describe the task and get back a draft llm eval dataset of {input, expected, rubric} cases…
SYNOPSIS
POST https://x402.agentutility.ai/eval-case-generate
Content-Type: application/json
X-PAYMENT: <signed-transferWithAuthorization>
{ ... }↳ first call →
402 Payment Required. Sign USDCtransferWithAuthorization, retry with theX-PAYMENT header.DESCRIPTION
Eval case generator for an agent or LLM task: describe the task and get back a draft llm eval dataset of {input, expected, rubric} cases spanning typical, edge, and adversarial scenarios. Pass example cases to steer style, and set n for how many to generate (default 8, max 20). Every case is labeled a candidate needing human review, not validated ground truth: this drafts a first pass at agent test cases, it doesn't certify them. Use it as an agent regression tests starter, an eval dataset generator for a new task, or a quick way to seed a rubric-graded benchmark before a human tightens it.
INPUT — request schema
| property | type | description | req? |
|---|---|---|---|
| task | string | The task or agent behavior to generate eval cases for, e.g. 'a customer-support agent that answers refund policy questions'. Required, max 4,000 chars. | required |
| examples | array | object | string | Optional example case(s) to steer style/format. JSON-serializable, capped at 20,000 chars when serialized. | optional |
| n | number | How many eval cases to generate. Integer, 1-20. Default 8. | optional |
OUTPUT — response shape
| field | type | description |
|---|---|---|
| task | string | — |
| requested_n | string | — |
| returned_n | string | — |
| cases | string | — |
| review_required | string | — |
| note | string | — |
| model | string | — |
| source | string | — |
EXAMPLES — two ways to call
EXAMPLE 1 · curl
curl -X POST https://x402.agentutility.ai/eval-case-generate \
-H 'Content-Type: application/json' \
-d '{ }'first response =
402 Payment Required with payment requirements; sign + retry with X-PAYMENT.EXAMPLE 2 · mcp
# Install the MCP package for this endpoint's cluster npx -y @agentutility/mcp-<cluster> # Required: EVM private key with USDC on Base export X402_PRIVATE_KEY=0x... # Then call the eval-case-generate tool from your MCP-aware agent.
MCP server handles payment automatically — your coding agent just calls the tool by name.
METADATA
- tags
- agentopsevalcasegenerateeval-case-generate
- methods
- POST
- cluster
- agentops
- price
- $0.02 USDC per call
ADJACENT — other endpoints in agentops
| endpoint | description | price |
|---|---|---|
| agent-trace-brief | Turns a raw agent execution trace into a readable agent trace summary: an ordered step-by-step account of what the agent did, where it br… | $0.02 |
| mcp-tool-risk-score | Heuristic MCP tool risk review for an MCP tool manifest before you wire it into an agent. | $0.02 |
| tool-call-diff | Deterministic tool call diff between two agent run sequences. | $0.008 |
SEE ALSO