Routing Agents with Jev and Laya: Adding System One Decisions to Quarkus LangChain4j

Lately, there’s been some buzz around Jev, TypeSafe’s “System One” model. The core idea is simple: instead of begging an LLM to output structured text and then writing a fragile parser to get that data back into your code, you send the model a state and some typed questions. It returns typed answers—complete with probability distributions and confidence scores—that your code can actually use for branching logic. No prompt-repair loops, no regex parsing, just direct values.

Then there’s Laya. Laya is the open-source (Apache 2.0) counterpart. It’s a System One decision model that speaks almost the exact same wire protocol as Jev, but you can host it yourself.

Both are interesting, but I wanted to see how they actually fit into a real project. I had a Quarkus app based on the Quarkus LangChain4j workshop that routes customer requests to one of four specialist agents. In the original version, routing was just another LLM chat call, and there was no real check on whether the specialists’ replies were actually any good or the right agent was called. This new decision model seemed like the perfect fit to improve this use case.

This post is essentially a “how-to” and a report on how much effort this took. The short answer? Not much. The framework already provides the two hooks I needed: a Planner to pick the agent and an OutputGuardrail to reject bad replies. Because these are plain interfaces, the whole feature ended up being suprisingly straightforward with only about 285 lines of actual decision logic. Everything else (the orchestration, retries, REST layer) is just Quarkus doing its thing.

One caveat: I couldn’t do a head-to-head Jev vs. Laya comparison because TypeSafe paused signups, so I don’t have an API key. My “Jev” runs are actually a deterministic stub, but the Laya runs are actually real. But since they share the same schema and interface, swapping them is just a matter of changing one config property.

What exactly is a System One model?

Standard LLMs are designed to talk to humans. When you try to make your code “consume” a judgment from an LLM, you’re fighting the tool: you coerce a text generator into acting like a database, then parse that text back into a type.

System One models on the other hand work by giving them a state (text, JSON, an email) and typed questions, and they return structured values. Jev and Laya both use three main primitives:

Primitive	The Question	What it returns
Choice	“Pick one from this list”	A choice, probabilities for each option, and confidence
Score	“Rate this on a scale”	A score, the legend, probabilities, and confidence
Noul	“Is this true?”	A yes-probability between `[0, 1]`

Because the answer is already typed, your code becomes: if (answer.choice() == "weather") .... No string parsing required.

I applied these primitives to two specific problems in my trip advisor app:

Routing: “Which specialist should handle this?” → a Choice question.
Reply Guardrail: “Does this reply actually answer the user?” → a Noul question.

The Setup

The base is a Quarkus service (Java 25, Quarkus 3.39.x, LangChain4j 1.14.x) that acts as a car-rental advisor. It initially used the SupervisorAgent pattern where a request comes in and an LLM decides which sub-agent(s) to call, like for example in our Quarkus LangChain4j Workshop.

My goal was to keep the shape of the supervisor but swap out the brain. Instead of a chat completion for routing, I wanted a calibrated decision model. And while I was at it, I wanted to add a guardrail check on the output—something the original app lacked.

Part One: The Routing Decision

The entry point for making agent routing decisions was the LLM SupervisorAgent, a built-in pattern in LangChain4j. Instead, I used the @PlannerAgent which allows you to plug in your own custom Planner (in this case a JevRoutingPlanner):

@PlannerAgent(
        name = "tripAdvisor",
        description = "Routes a customer request to the right specialist using a Jev decision",
        outputKey = "reply",
        subAgents = { ReservationAgent.class, WeatherAgent.class, CostAgent.class, GeneralAgent.class })
String planTrip(String request);

@PlannerSupplier
static Planner planner() {
    return new JevRoutingPlanner();
}

The JevRoutingPlanner reads the customer message and asks the decision model one Choice question. It then returns an action to invoke the matching sub-agent.

@Override
public Action firstAction(PlanningContext context) {
    AgenticScope scope = context.agenticScope();
    String request = readRequest(scope);
    if (request == null || request.isBlank()) {
        request = "general question";
    }

    JevRouter router = new JevRouter(decisionClient(), routeAudit());
    JevRouter.RouteDecision decision = router.route(request);
    scope.writeState("route", decision.route());
    scope.writeState("rawChoice", decision.rawChoice());

    return call(pickSubagent(decision));
}

The logic in JevRouter is also pretty straightforward: I define a map of criteria for each specialist, and the model picks the best fit.

public RouteDecision route(String request) {
    Map<String, String> criteria = new LinkedHashMap<>();
    criteria.put(ROUTE_RESERVATION, "Booking, modifying, cancelling, or questions about a rental reservation or pick-up");
    criteria.put(ROUTE_WEATHER, "Weather, forecast, rain, snow, or climate for the trip or destination");
    criteria.put(ROUTE_COST, "Pricing, total cost, budget, fees, or cost comparison");
    criteria.put(ROUTE_GENERAL, "Anything else, or a greeting or general question about the service");

    String chosen = decision.choose(request, "route",
            JevQuestion.choice("Which specialist should handle this customer request?", criteria));
    RouteDecision decision = decide(chosen);
    routeAudit.log(request, decision.route(), decision.rawChoice());
    return decision;
}

The mapping from the model’s raw string back to a route is a simple Java switch statement:

public static RouteDecision decide(String chosen) {
    String route = switch (chosen) {
        case ROUTE_WEATHER -> ROUTE_WEATHER;
        case ROUTE_COST -> ROUTE_COST;
        case ROUTE_RESERVATION -> ROUTE_RESERVATION;
        case null, default -> ROUTE_GENERAL;   // unknown -> safe default
    };
    return new RouteDecision(route, chosen);
}

Part Two: The Reply Guardrail

These decision models are also a good fit for guardrails: Before a specialist’s reply goes back to the user, an OutputGuardrail asks a Noul question: “Does the drafted reply directly address the customer’s request?”

I used a “margin” approach for the decision:

Clearly above 0.5: Success.
Clearly below 0.5: Retry (with guidance to re-answer the actual question, concisely).
In the middle (the margin): Pass, but flag it for human review.

The margin is 0.1, so a near-50/50 answer doesn’t drive an automatic retry, but a confident “no” does. Here is the whole validation method:

@Override
public OutputGuardrailResult validate(OutputGuardrailRequest request) {
    String text = request.responseFromLLM().aiMessage().text();
    if (text == null || text.isBlank()) {
        return success();
    }
    String customerRequest = String.valueOf(request.requestParams().variables().get("request"));

    String state = "Customer request: " + customerRequest + "\n\nDrafted reply: " + text;
    Double addressIt = client().noul(state, "addresses_request",
            JevQuestion.noul("Does the drafted reply directly address the customer's request?"));

    if (addressIt == null) {
        return success();                             // no answer: pass, flag for review
    }
    if (addressIt < 0.5 - DECISION_MARGIN) {          // clearly below 0.5
        return retry("The reply does not directly address the customer's request. "
                + "Re-answer using only the customer's actual question and keep it concise.");
    }
    if (addressIt > 0.5 + DECISION_MARGIN) {          // clearly above 0.5
        return success();
    }
    return success();                                 // within the margin: pass, flag for review
}

This is trivial to plug in to the agent with the OutputGuardrails feature, which also has a built in way to retry:

@Agent(description = "Answers weather and forecast questions for the trip or destination",
       outputKey = "reply")
@OutputGuardrails(value = JevReplyGuardrail.class, maxRetries = 2)
String answer(String request);

One Interface, Three Backends

I created a generic DecisionClient interface for the router and guardrail to use, with three different implementations:

Jev: The hosted TypeSafe endpoint (needs an API key).
Laya: A self-hosted sidecar (open source, Apache 2.0).
Stub: A deterministic offline version for testing.

A single CDI bean picks the delegate from one property:

String backend = config.getOptionalValue("decision.backend", String.class).orElse("jev");
this.delegate = switch (backend.toLowerCase()) {
    case "laya" -> laya;
    case "stub" -> new StubDecisionClient();
    case "jev" -> jev;
    default -> { Log.warnf("Unknown decision.backend '%s'; using 'jev'", backend); yield jev; }
};

Since Jev and Laya share the same wire protocol, swapping them is a one-line change in your config. This is huge for PoCs—you can build everything on a self-hosted model and only move to a paid endpoint if it actually proves its worth on your specific data.

Real-World Results

Laya in Action

I ran Laya behind a small FastAPI sidecar using the convaiinnovations/laya-typed-decisions checkpoint. (Note: the base checkpoints are pretty weak; you really need the fine-tuned version for this to work.)

cd laya-sidecar
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
uvicorn main:app --host 0.0.0.0 --port 8100

Then point the app at it:

export OPENAI_API_KEY=sk-...
./mvnw "-Ddecision.backend=laya" quarkus:dev

The backend endpoint confirms which model is active:

$ curl -s http://localhost:8083/trip/backend
{"backend":"laya","model":"laya"}

For straightforward requests, Laya routed perfectly, and the LLM wrote the reply:

$ curl -s -X POST http://localhost:8083/trip -H 'Content-Type: application/json' \
    -d '{"request":"Will it rain in Lisbon next Tuesday?"}'

{
  "request": "Will it rain in Lisbon next Tuesday?",
  "reply": "As of the latest forecast, Lisbon is expected to experience rain next Tuesday. " +
           "If you're planning to drive, be prepared for potentially wet and slippery road conditions. " +
           "It's a good idea to ensure your rental car's windshield wipers are functioning well and " +
           "to allow extra travel time to accommodate any slower traffic due to the weather. Safe travels!",
  "route": "weather",
  "rawChoice": "weather",
  "backend": "laya",
  "model": "laya",
  "live": true
}

$ curl -s -X POST http://localhost:8083/trip -H 'Content-Type: application/json' \
    -d '{"request":"How much does it cost to rent an SUV for 5 days?"}'

{
  "request": "How much does it cost to rent an SUV for 5 days?",
  "reply": "To provide a detailed estimate for renting an SUV for 5 days, let's break down the costs " +
           "into base rates, optional extras, and fees. ... Estimated total cost: $475 to $975 ...",
  "route": "cost",
  "rawChoice": "cost",
  "backend": "laya",
  "model": "laya",
  "live": true
}

$ curl -s -X POST http://localhost:8083/trip -H 'Content-Type: application/json' \
    -d '{"request":"Please reserve a car for next week in Paris"}'

{
  "request": "Please reserve a car for next week in Paris",
  "reply": "Certainly! I can help you with that. Could you please provide the following details to " +
           "complete your reservation? 1. Pick-up Date and Time ... 2. Drop-off Date and Time ...",
  "route": "reservation",
  "rawChoice": "reservation",
  "backend": "laya",
  "model": "laya",
  "live": true
}

$ curl -s -X POST http://localhost:8083/trip -H 'Content-Type: application/json' \
    -d '{"request":"Hi there!"}'

{
  "request": "Hi there!",
  "reply": "Of course! Please go ahead and ask your question, and I'll do my best to assist you.",
  "route": "general",
  "rawChoice": "general",
  "backend": "laya",
  "model": "laya",
  "live": true
}

But the real value shows up when things get ambiguous.

Example: “I want to cancel my reservation, how much is the cancellation fee?”

Laya picked reservation (0.52), but cost was a close second (0.35). Crucially, the confidence dropped to 0.25:

{
  "choice": "reservation",
  "probabilities": {
    "reservation": 0.5247,
    "weather": 0.0434,
    "cost": 0.3506,
    "general": 0.0813
  },
  "confidence": 0.2454
}

A standard chat model would just pick one and act confident. Laya on the other hand basically tells me: “I’m picking reservation, but I’m actually not very sure.” This kind of signal is pretty interesting, as it allows you to trigger a clarifying question or escalate to a human.

Two more mixed inputs, both with very low confidence (0.07 and 0.05):

What's the weather like in Rome and how much is a convertible there?

{ "choice": "weather", "probabilities": { "reservation": 0.1682, "weather": 0.4416, "cost": 0.2435, "general": 0.1467 }, "confidence": 0.0721 }

Do you have a discount for long-term rentals in the summer?

{ "choice": "reservation", "probabilities": { "reservation": 0.3764, "weather": 0.1278, "cost": 0.2279, "general": 0.2679 }, "confidence": 0.0474 }

As you can see, there’s high confidence on clear inputs and explicit low confidence on the ambiguous ones. This is the behavior you want from a calibrated decision model, instead of a chat model that commits to an answer either way.

Final Verdict: Is it worth it?

For routing, absolutely. If you have a fixed set of buckets, a System One model is a better fit than a chat-based supervisor: you skip the chat inference and the parsing step on every request, and you get quantified uncertainty instead of a model committing to prose.

A few warnings, though:

High cardinality: If you have 30+ specialists, Laya might struggle (the options share a fixed token budget); Jev is generally stronger in high-option spaces.
Noul and Score primitives: These are the weakest parts of these models. They need careful thresholds and a good checkpointthe guardrail only worked cleanly on the fine-tuned one.

The biggest win for me was the integration. The feature is small, reversible (basically change the annotation from @PlannerAgent back to @SupervisorAgent you’re back on the built-in supervisor), and testable without an internet connection. And because Laya is Apache 2.0 and wire-compatible with Jev, you have a real path to ownership over your decision layer: build on the open model, and only pay for the hosted endpoint if it beats it on your data.

Key files

Path	Role
`agentic/TripAdvisorSystem`	top-level `@PlannerAgent` wiring the four sub-agents
`agentic/JevRoutingPlanner`	custom `Planner` — the decision-model router
`agentic/JevRouter`	request → `Choice` question → normalized route
`jev/DecisionClient`	the backend-agnostic interface everything depends on
`jev/ActiveDecisionClient`	picks the backend from `decision.backend`
`jev/JevApiClient` / `LayaDecisionClient` / `StubDecisionClient`	the three backends
`guardrails/JevReplyGuardrail`	the Noul-driven output guardrail
`laya-sidecar/main.py`	the ~90-line FastAPI host for Laya