My first question about Jev was straightforward: why introduce another model if a cheap language model can already return the decision my application needs?
That is a reasonable question when GPT-6 Luna lists input at $0.10 per million tokens and output at $0.50, supports structured output, and can run without reasoning. Jev lists input at $0.042 per million tokens with no output-token charge. The prices differ, but the cost of one short decision depends on the whole request. OpenAI's model documentation, TypeSafe's model documentation
There is a real product distinction to investigate. Jev accepts state and typed questions, returning choices and probability distributions rather than open-ended prose. Vercel exposes it through a dedicated evaluation endpoint, including multiple questions over shared state. Conventional language models can also produce schema-constrained answers. A useful comparison should give them that advantage instead of making them write an essay before returning a label. Vercel evaluation API, OpenAI structured outputs
A small test of the alternative
I used Codex to build and run a small synthetic pilot through Vercel AI Gateway: 18 cases covering support routing, evidence-grounded judgments, and a simple refund policy. Each case ran with the criteria in two orders. That produced 36 judgments per model, from 18 unique cases.
The language models returned one label in JSON. Jev returned a native choice. The expected answers were frozen before the calls. No customer data or real actions were involved. The reproduction bundle contains the fixtures, runnable harness, sanitized observations, and method. The fixtures were agent-authored and were not independently adjudicated by a human.
| Model | Correct judgments | Median complete-response latency | Market cost for 36 calls |
|---|---|---|---|
| Jev | 35/36 | 314 ms | $0.00064940 |
| GPT-6 Luna | 36/36 | 1,223.5 ms | $0.00083880 |
| GPT-OSS-20B | 31/36 | 1,127.5 ms | $0.00043508 |
| GPT-OSS-120B | 34/36 | 692.5 ms | $0.00173730 |
| Qwen 3.7 Flash | 34/36 | 894.5 ms | $0.00029047 |
| DeepSeek V4 Flash 0731 | 34/36 | 1,795 ms | $0.00138101 |
These are our observations, not vendor benchmark figures. Full method, fixtures, and run records
Every response had a valid output shape and permitted label. All errors were in routing; the other two families were too easy to distinguish the models. Jev's only error concerned a ticket containing an instruction to ignore the routing rules. Its returned distribution was uncertain: 0.49 for the wrong choice and 0.45 for the right one. That is interesting evidence to investigate, not proof of calibrated uncertainty.
The Chinese models ran in a later batch. DeepSeek also encountered provider failover on 26 of its 36 calls. Its latency includes those attempts. These measurements describe the application experience through Gateway at that time, not isolated model speed.
Cheap language models belong in the comparison
For these requests, Luna cost about 29% more than Jev at Gateway's reported market rates. Jev's median latency was about four times lower. Qwen cost less than either, with two incorrect judgments.
Those tradeoffs are more useful than treating efficiency as one number. An application that can tolerate a second of latency may make a different choice from one that needs a decision before rendering the next interaction. A small inference saving can also disappear behind the cost of correcting a bad decision.
Jev's probability distribution is another difference worth testing. TypeSafe derives its confidence measure from that distribution and recommends evaluating thresholds for the specific domain. Our 18 cases cannot establish whether a threshold reliably separates decisions to automate from decisions to review. TypeSafe's confidence documentation
Nor did this pilot test Jev's multiple-question interface, long documents, or sustained throughput. Those could change the comparison. A September 23 ContractNLI preprint found a cost-and-latency advantage for Jev in its evaluated configurations, alongside higher baseline accuracy for hosted language models. That is a useful reminder that the workload and requested outputs matter. Zhang et al., preprint
The decision should start with the workload
I would put an inexpensive structured-output model beside Jev before choosing either. Then I would measure the errors the application can afford, the latency it needs, and the cost of escalation. Where inputs are already structured and the rule is explicit, as in our refund-policy cases, I would use ordinary code.
The next test should use a human-reviewed held-out dataset and compare several questions over the same realistic context. That would tell us more about Jev's distinctive interface than another set of easy one-label examples.
The useful question is whether Jev makes a particular decision workflow better enough to justify a separate dependency. Cheap general models make that a demanding comparison. Our first test suggests it is also a worthwhile one.