Intent routing and model cascades

Choose between deterministic code, a specialist LLM, and a human without sending every request to the most expensive handler.

CHOICE×1SCORE×1

One request, 2 parallel answers. The model never performs the side effect — your code reads the confidences and decides.

QUESTION DESIGN

questions = {
    "intent": Choice(
        instructions="What is the user's primary request?",
        criteria={
            "order_status": "The user wants the status of an existing order.",
            "product_question": "The user wants product information or advice.",
            "return_exchange": "The user wants to return or exchange an item.",
            "complaint": "The user reports a service failure or dissatisfaction.",
            "other": "No route clearly fits.",
        },
    ),
    "complexity": Score(
        instructions="How difficult would it be for a support system to resolve this request safely?",
        criteria=[
            "A deterministic lookup or short answer is enough",
            "A specialist model and some context are needed",
            "Multiple records, policy interpretation, or a human decision is needed",
        ],
    ),
}

Ask for the route and the difficulty separately. A high-confidence intent can still be too complex for automation.

1 次请求 / 2 个答案

这个模式的一句话

ROUTING POLICY

def choose_handler(response) -> str:
    intent = response.answers["intent"]
    complexity = response.answers["complexity"]

    if intent.confidence < 0.5 or complexity.confidence < 0.5:
        return "human"

    if intent.choice == "order_status" and complexity.score < 1:
        return "order_database"
    if intent.choice == "return_exchange":
        return "returns_specialist"
    if intent.choice == "complaint" and complexity.score >= 1:
        return "human"
    if intent.choice in {"product_question", "complaint"}:
        return "specialist_llm"
    return "human"

The route is an application decision. Jev does not grant access to the selected handler, and it should never bypass authentication or authorization.

Source: Intent routing.