Jev in production › Evaluation and testing

Pinecone Cultivar

Skill-testing CLI with an optional Jev grader (--backend typesafe) that judges each agent run against plain-English task criteria and returns a calibrated pass probability and failure kind. ~30x cheaper than Claude grading (company).

Open on GitHub ↗

~30x cheapermeasured against a baseline, as published by the source
Use
Evaluation and testing
Industry
Developer tools
Form
Open-source tool
Stage
In production
Plugs into
Claude Code
Listed
2026-09-19
Found via
github
Repository
pinecone-io/cultivar
Stars
42
Forks
3
Last push
2026-10-09
Language
Python
License
MIT

The README opens with

A CLI tool to help you write tests for skills, test them across agents, and iterate until they work, from the Pinecone DevRel team.

Test how well skills work against tasks, across agents, locally and remotely. Customize sandboxes for how agents should start, and graders for how agents should work.

Use traces to iteratively refine skills and optimize them against tasks.

Benchmark against skills, docs, and baselines. And, even run in parallel simulatenously for faster execution.

Badge

For the project's own README, linking back here:

Listed in Jev in production

Also used for evaluation and testing