Jev Playground
benchmarks Jev against Luna, Haiku, and Gemini at choosing validated legal moves in explicit-state games, scoring decision quality and consistency across a sequence of moves.
何を判断するか
CHOICE
Picks one option from a fixed set
SCORE
Rates against ordered levels
NOUL
Answers a yes / no question
Inferred from this project's own one-line summary in the community list — it reads as a CHOICE + SCORE + NOUL decision. Read the source to confirm.
CHOICE 呼び出しの形
この質問型の一般的な骨格であって、このプロジェクトの実際のコードではない。
from typesafe import TypeSafe
ts = TypeSafe() # reads TYPESAFE_API_KEY
result = ts.evaluate(
state=page_state,
questions={
"next_action": {
"type": "choice",
"options": ["click", "scroll", "type", "done"],
"instructions": "What should the agent do next?",
}
},
)
action = result["next_action"] # the chosen option
confidence = result["next_action_confidence"]評価とベンチマークの他のプロジェクト
Jev Web Analyzeranalyzes a public SaaS landing page as clean Markdown and asks Jev ten bounded `Choice` questions about first-visit understanding, returning inspectable findings for the first change to make.Jev vs Mistral and Gemini for event validationhead-to-head test of Jev against Mistral Small and Gemini Flash-Lite at validating local event listings.jev-research-evalreproducible eval harness plus field note for Jev Ultrafast research-browser tasks, with QC'd cases, a suite runner, and a report generator.Jev judge call vs dimension scorestests one direct Jev question per row against 12–14 Jev-scored dimensions with locally fitted weights on three classification tasks, reaching 0.9076 against 0.8373 on Japanese NLI but flagging about 25× more hard benign rows as attacks.