LegalForecast-MTD

benchmark that asks Jev to predict federal motion-to-dismiss rulings from the judge's written record and scores the calibrated probabilities with claim-defendant micro-Brier metrics.

THE SHAPE OF A SCORE CALL

A generic skeleton for this question type, not this project's actual code.

from typesafe import TypeSafe

ts = TypeSafe()
result = ts.evaluate(
    state=candidate,
    questions={
        "relevance": {
            "type": "score",
            "levels": ["low", "medium", "high"],
            "instructions": "How relevant is this to the query?",
        }
    },
)
if result["relevance"] >= 0.7:
    keep(candidate)