📄 Answer Completeness¶

metrics.llm_as_judge.direct.criteria.answer_completeness

CriteriaWithOptions(
    name="answer_completeness",
    description="The response is complete: all the aspects of the reference answer are addressed in the response. The response might use different phrasing or wording from the reference answer.",
    prediction_field=None,
    context_fields=None,
    options=[
        CriteriaOption(
            name="Excellent",
            description="The response addresses all aspects of the reference answer.",
        ),
        CriteriaOption(
            name="Good",
            description="The response addresses most aspects of the reference answer, with minor omissions.",
        ),
        CriteriaOption(
            name="mediocre",
            description="The response covers the essential aspects of the reference answer but has notable omissions.",
        ),
        CriteriaOption(
            name="Bad",
            description="The response covers only a few aspects of the reference answer, with significant omissions.",
        ),
        CriteriaOption(
            name="Very Bad",
            description="The response fails to address the reference answer meaningfully, with most aspects omitted.",
        ),
    ],
    option_map={
        "Excellent": 1.0,
        "Good": 0.75,
        "mediocre": 0.5,
        "Bad": 0.25,
        "Very Bad": 0,
    },
)

[source]

from unitxt.llm_as_judge_constants import CriteriaOption

Explanation about CriteriaWithOptions¶

Criteria used by DirectLLMJudge to run evaluations.

Explanation about CriteriaOption¶

A criteria option.