NodeLLM 1.18: Judgments with Jev and Typed Structured Output

A lot of the LLM calls I see in real apps aren't about generating text at all. Is this ticket urgent? Which team should handle it? How angry is this customer? We ask a chat model, tell it to reply with JSON, parse that, and then hope "high" means the same thing every time.

What we actually want in these cases is a decision our code can branch on. That's what NodeLLM 1.18 adds: judgments, powered by TypeSafe's Jev models.

⚖️ Judgments: Ask Questions, Get Numbers

Jev models don't write text. You give them an input and some typed questions, and they return a probability, a choice or a score for each one.

import { NodeLLM, probability } from "@node-llm/core";

const judgment = await NodeLLM.judge("Please refund the duplicate charge today.", {
  questions: {
    urgent: probability("Does this need attention today?")
  }
});

judgment.urgent.probability; // 0.91

Set TYPESAFE_API_KEY (or JEV_API_KEY) and that's all the setup. The model defaults to jev-latest, and judgments route to TypeSafe based on the model, so this works from any NodeLLM instance, whichever chat provider it's set up for.

Three Kinds of Question

There are three question builders: probability(), choice() and score(). All of them are answered in one request, over the same input:

import { NodeLLM, probability, choice, score } from "@node-llm/core";

const judgment = await NodeLLM.judge(ticket.body, {
  questions: {
    urgent: probability("Does this need attention today?"),

    department: choice("Which team should handle this?", {
      billing: "Payments and refunds",
      technical: "Bugs and integrations",
      other: null
    }),

    frustration: score("How frustrated is the customer?", ["Calm", "Frustrated", "Angry"])
  }
});

judgment.department.choice;        // "billing" | "technical" | "other"
judgment.department.probabilities; // { billing: 1, technical: 0, other: 0 }
judgment.frustration.score;        // 0 to 2, e.g. 0.83

The answers are typed from your questions, so department.choice is a union of your option keys, not a string. The thresholds stay in your code, where they belong:

if (judgment.urgent.probability >= 0.8) {
  await ticket.update({ priority: "high" });
}

One thing to keep in mind: a probability near 0.5 means "yes and no are about equally likely", not "medium urgency". If you want to measure how much of something there is, use a score.

Reusable Judges

If you've used the Agent class, this will feel familiar. You can declare questions once on a Judge class and reuse them:

import { Judge, probability, choice } from "@node-llm/core";

class TicketTriage extends Judge {
  static questions = {
    urgent: probability("Does this need attention today?"),
    department: choice("Which team should handle this?", {
      billing: "Payments and refunds",
      technical: "Bugs and integrations",
      other: null
    })
  };
}

const judgment = await TicketTriage.judge(ticket.body);

There's also defineJudge() if you'd rather not write a class, and questions can be built from runtime data, for example a list of teams loaded from your database.

Jev costs $0.042 per million input tokens, and output is free. judgment.usage reports tokens and cost the same way chat responses do. If you run a Jev-compatible server yourself, point typesafeApiBase at it.

🧩 Typed Structured Output

This one has been on my list for a while. withSchema() now infers the type from your Zod schema, so response.data is typed instead of unknown:

const Person = z.object({ name: z.string(), age: z.number() });

const response = await NodeLLM.chat().withSchema(Person).ask("Extract the person.");

response.data.name; // string

This works for Chat#ask() and for Agent#ask() / say(). One small breaking change comes with it: writing const chat: Chat<Person> = NodeLLM.chat() no longer compiles. Call withSchema() and let it infer the type.

Getting Started

npm install @node-llm/core@1.18.0 @node-llm/mcp@0.3.0 @node-llm/testing@0.6.0

The Judgments guide has everything else. For the full list of changes, see the CHANGELOG.