Jev in production › Model and agent routing

jevcache (kushals256)

Local OpenAI-compatible proxy in which Jev judges whether a new prompt has the same intent as a cached one and serves the stored answer on a match, failing open to the model. A Jev-confirmed hit returned in 394ms, against 3244ms for a miss (author).

Open on GitHub ↗

394msmeasured against a baseline, as published by the source
Use
Model and agent routing
Industry
AI infrastructure
Form
Open-source tool
Stage
In production
Listed
2026-09-22
Found via
github
Repository
kushals256/jevcache
Stars
12
Forks
0
Last push
2026-10-03
Language
TypeScript
License
MIT
jevcache (kushals256) screenshot
docs/demo.gif in the kushals256/jevcache README, MIT; shown from GitHub.

The README opens with

MorrowCache — routers pick a model. This proxy decides whether to call one.

npm package and CLI stay @kushalicious/jevcache / jevcache (download history unchanged). Site: morrowcache.vercel.app.

A local OpenAI-compatible chat response cache. 2.0 (Agent Turn Cache) skips the model in two cases: the same question in different words, and an agent retrying the same job (same prior conversation, same-intent latest ask). It stores final assistant text only. A later stream of that answer can HIT. A response that contains toolcalls is returned and never stored.

Badge

For the project's own README, linking back here:

Listed in Jev in production

Also used for model and agent routing