Jev in production › Evaluation and testing

jevotron

Scores agent trace steps with Jev to detect anomalies and harmful steps; a 24-trace pilot matched 130 of 163 published step-quality labels, 79.8% agreement (author).

Open on GitHub ↗

79.8%measured against a baseline, as published by the source
Use
Evaluation and testing
Industry
Developer tools
Form
Open-source tool
Stage
Pilot
Listed
2026-10-09
Found via
github
Repository
cmungall/jevotron
Stars
12
Forks
2
Last push
2026-10-11
Language
Python
License
BSD-3-Clause

The README opens with

Find the fields worth a second look. A CLI for field-level anomaly detection in tabular, structured, and text files, SQLite, and DuckDB, powered by Jev.

Preview the input, scan every entry, and export a focused review queue. Each chunk is independent and sees the same optional guidance and exemplars. SQLite caching reuses assessments across file versions.

For large guidance documents, opt into token-aware packing with --batch-size auto. Batching controls and limits explain when it saves tokens and how to compare scores with per-entry requests.

Badge

For the project's own README, linking back here:

Listed in Jev in production

Also used for evaluation and testing