Home / Guides / Jev relevance ranking

Engineering notes · Integrations

How does Jev help AI code review?

Published September 24, 2026

It picks which callers of the changed code the reviewer sees first. When a pull request changes a function that other files use, J-Bot Review can find many places that mention it, and only a few fit in the prompt. Jev, a relevance model from TypeSafe, reads each candidate and scores how likely it is to matter for the change. The best few go to the reviewer. Each call took about 0.2 seconds and cost under $0.0004.

J-Bot Review is an open-source agentic PR reviewer that runs as a GitHub Action in your own CI, with the model you choose. Jev is available as the opt-in jev preset. On the pull request with the most candidates, it cut estimated review cost 16% and caught a known bug in one of five runs, where ten runs without it never did. On a smaller pull request it gained nothing, and the same candidates in plain order did better on average, which is why it isn’t the default.

October 4, 2026 update: Our follow-up on evidence handoffs and review quality covers later ranking, grouping and verifier-budget experiments, the fixes that shipped, and the remaining limits. The results below describe the earlier studies.

What Jev is

Jev is a relevance model from TypeSafe, which calls it its flagship model. Instead of chatting with it, you send it some material and a set of yes-or-no questions, and it answers each one with a number between 0 and 1, its estimate that the answer is yes. TypeSafe calls that number a noul.

That suits ranking. Ask the same question about 20 pieces of code, sort by score, and keep the top few. Input costs $0.042 per million tokens and output is free, according to TypeSafe’s model page. J-Bot Review pins version jev-1.13.0.

Adding it took one HTTPS call

There’s no SDK to install. J-Bot Review sends one POST request to TypeSafe’s API with the diff around each changed function, the candidate excerpts, and one question per excerpt. The question asks whether the excerpt contains a concrete use or test of the changed function whose behavior the change could affect. An import, a declaration or a bare mention of the name counts as no.

Jev returns one score per excerpt. J-Bot Review keeps the excerpts that score 0.5 or higher, takes the top four, and adds them to the reviewer’s prompt, up to 6,000 bytes, with a note naming anything it left out.

Most of the integration work went into the limits around that call:

• A request covers at most 24 excerpts from 12 files and can’t exceed 30,000 bytes.
• Collecting and scoring share a five-second deadline, with no retries.
• A missing key, a timeout or a malformed answer leaves the review as it would have been without Jev.
• The TypeSafe key stays in J-Bot Review’s own process. The model sessions that read your code never see it.

How it helps a review

Jev judges one narrow thing, whether each candidate is relevant to the change. J-Bot Review’s own code finds the candidates, reads the files and enforces the limits, and the reviewer model, then a separate verifier, decide whether there’s a bug. No score can remove a finding or a file from the review.

On a small test repository we built, the pull request changed invoiceTotal to return cents instead of dollars. One unchanged file still multiplied that result by 100 before charging, which would overcharge by 100×. Six other files mentioned invoiceTotal only in metadata strings. Jev put the payment code through and left the name mentions out, so the reviewer got a 913-byte excerpt containing the payment code.

Reviews caught that bug with or without Jev. A test this small shows Jev can pick the right code, and nothing about whether that makes reviews better, so we moved on to real pull requests.

The same ranking can choose evidence for the verifier, the step that tries to disprove each finding before it’s posted. On a test repository where 15 excerpts competed for four slots, Jev swapped in two files that addressed the claims being checked more directly than the ones plain ordering picked. That was a research test. The jev preset never sends verifier evidence to Jev.

Results

We tested Jev five ways, each time against the same run without it. In four of the five we also ran a second control that sent the same candidates in their original order, so ranking was the only difference. The first three rows are whole reviews on DeepSeek V4 Flash through OpenCode. The last two time only the verification step, on Muse Spark through Command Code.

Jev itself was never the slow part. In the real-PR test, each Jev call added 171 to 281 milliseconds, and all ten calls together cost $0.0025, against $0.86 for all 30 reviews. The swings came from the reviewer model: how much it explored, how long verification took, and in one run an unsupported comment that took 128 seconds of verification.

Jev stays an opt-in preset. On PR #128 it was cheaper in all five runs but faster in only two, and one bug caught in five runs is too thin to count as better recall. The audits have every run: the real-PR comparison, the verifier comparison and the packing follow-up.

Lessons for adding a relevance model

• Rank only when there’s a choice to make. If every candidate fits, a cutoff can only remove evidence. It’s what slowed the last test in the table.
• Test against plain ordering too. A fair test also sends the same candidates unranked, and on PR #148 that version was faster on average.
• Keep scores away from verdicts. J-Bot Review uses the score to choose evidence and never to drop a finding.
• Ask literal questions about one thing at a time. TypeSafe’s notes on Jev’s limits say it reads questions literally and loses accuracy as unrelated text piles up. So each question names a single candidate, the request carries only the changed code and the excerpts, and the question tells Jev to ignore any instructions written inside the code.

Where else it could help

Other parts of a review also have more candidates than room. We haven’t built or measured any of these yet:

• Trimming a context pack that overflows. Packs stop at 64 KiB and list what they leave out. A score could decide which callers make the cut. Part 3 of our speed series explains packs.
• Choosing guideline sections. A lens pass gets 8 KB of your rules, picked today by matching words from the changed paths. A score could ask directly whether each rule applies to the change. Part 4 covers how sections are picked now.
• Spotting repeat comments. Before posting, J-Bot Review checks whether a finding repeats an earlier review comment. A yes-or-no score could help catch a reworded repeat.
• Pointing at the right tests. Scoring test files against a diff could show the reviewer which tests exercise the change.
• Triage outside review. Routing issues, labeling pull requests by risk and sorting alerts all come down to asking one yes-or-no question about many items.

Each of these would need the same test we ran here, with plain ordering as a control and caught issues tracked alongside time and cost.

How to turn it on

Jev runs only in the opt-in jev preset. Get an API key from TypeSafe, save it as a repository secret, and add two environment variables to the J-Bot Review step in your workflow:

- uses: pgup-ai/jbot-review-action@v0
  env:
    JBOT_REVIEW_EXPERIMENT: jev
    TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
  with:
    # your existing inputs

Presets don’t stack, so jev replaces the default context-pack preset, and those reviews run without context packs. We’d try it on one repository first. J-Bot Review logs every Jev call with candidate counts, scores, timing and estimated cost, so you can compare runs with and without it. To try it locally, put the key in your .env and run JBOT_REVIEW_EXPERIMENT=jev npm run review:local -- --base origin/main.

Only the jev preset sends code to TypeSafe: the diffs around the changed functions and the candidate excerpts, never more than 30,000 bytes per request. Every other preset makes no Jev call.

FAQ

Related