Scorers and judge models

A scorer turns a trace into an assessment. Some are deterministic; some use a judge model, which calls through the same gateway as agents and under an allowlist your administrator controls.

Under construction

This page is planned and not yet written, because the capability it describes is not yet generally available. The structure is here so you can see where it fits; the notice comes off when it ships.

What this page will cover

• The built-in scorer library.
• Configuring a judge model and its allowlist.
• Publishing a tenant-authored scorer without shipping code.
• Calibrating a judge before a gate enforces.

Meanwhile

• Evaluation overview explains where scorers sit in the flow.