Jev vs LLMs on support ticket triage

Can a System One model (TypeSafe Jev) route support tickets as accurately as general-purpose LLMs, for less money and in less time? And can its confidence decide which tickets are safe to automate?

Headline

Automate the confident ones

Rank tickets by how confident each system is and hand the most confident ones to automation. Pick the accuracy you need on the automated share. Each bar shows how many tickets clear that bar, and the rest go to a human.

90%

    Accuracy on the most-confident share of tickets. A curve that stays high further to the right means the confidence score is more useful. The horizontal rule marks the required accuracy.

    Cost and speed

    Measured from the token usage each API returned, multiplied by the listed price. Latency is wall-clock time per call, including network time.

    Cost per 1,000 tickets (USD)

    Median latency per ticket

    Ticket explorer

    All numbers

    Every value above, as a table. Where a system was run 3 times, accuracy is the mean with the range in brackets.

    Method and caveats