Datasets and experiments

A dataset is a versioned set of examples with expected outcomes. An experiment grades a candidate against a dataset version and produces aggregate metrics you can compare side by side.

Under construction

This page is planned and not yet written, because the capability it describes is not yet generally available. The structure is here so you can see where it fits; the notice comes off when it ships.

What this page will cover

• Create a dataset from production traces or from a file.
• Version a dataset and reference it from a target.
• Run an experiment and read its aggregate metrics.
• Compare experiments side by side.

Meanwhile

• Evaluate a version already covers pinning a target to a dataset version and comparing with a baseline.