Jevals.com
independent leaderboard that asks Jev and six LLMs the same Noul, Choice and Score questions and grades every answer against human labels (PubMedQA, Banking77, HelpSteer2; 300 items × 5 runs each), finding Jev tied for first on PubMedQA yes/no at 1/28 of the top LLM's price, tied for second on Banking77 and no model beating the label base rates on HelpSteer2, with every per-decision probability published as CC BY 4.0 data.
WHAT IT DECIDES
CHOICE
Picks one option from a fixed set
SCORE
Rates against ordered levels
NOUL
Answers a yes / no question
Inferred from this project's own one-line summary in the community list — it reads as a SCORE + NOUL decision. Read the source to confirm.
MORE IN Evaluation & Benchmarking
Jev Web Analyzeranalyzes a public SaaS landing page as clean Markdown and asks Jev ten bounded `Choice` questions about first-visit understanding, returning inspectable findings for the first change to make.Jev Playgroundbenchmarks Jev against Luna, Haiku, and Gemini at choosing validated legal moves in explicit-state games, scoring decision quality and consistency across a sequence of moves.Jev vs Mistral and Gemini for event validationhead-to-head test of Jev against Mistral Small and Gemini Flash-Lite at validating local event listings.jev-research-evalreproducible eval harness plus field note for Jev Ultrafast research-browser tasks, with QC'd cases, a suite runner, and a report generator.