Type alias Criteria

Configuration to load a "CriteriaEvalChain" evaluator, which prompts an LLM to determine whether the model's prediction complies with the provided criteria.

Param: criteria

The criteria to use for the evaluator.

Param: llm

The language model to use for the evaluator.

Returns

The configuration for the evaluator.

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "criteria",
    criteria: "helpfulness"
  }]
};
Copy

Example

const evalConfig = {
  evaluators: [{
    evaluatorType: "criteria",
    criteria: { "isCompliant": "Does the submission comply with the requirements of XYZ"
  }]
};
Copy