📄 Logical Validity Of Reasoning¶

metrics.llm_as_judge.direct.criteria.logical_validity_of_reasoning

CriteriaWithOptions(
    name="logical_validity_of_reasoning",
    description="Assess whether the model reasoning is logically valid when solving problems in propositional logic. The reasoning should follow correct logical principles and lead to a valid conclusion based on the given premises.",
    prediction_field="model reasoning",
    context_fields=[
        "problem statement",
        "statements",
    ],
    options=[
        CriteriaOption(
            name="Yes",
            description="The reasoning is logically valid and correctly applies propositional logic principles.",
        ),
        CriteriaOption(
            name="No",
            description="The reasoning is logically invalid or contains errors in applying propositional logic principles.",
        ),
    ],
    option_map={
        "Yes": 1.0,
        "No": 0.0,
    },
)

[source]

from unitxt.llm_as_judge_constants import CriteriaOption

Explanation about CriteriaWithOptions¶

Criteria used by DirectLLMJudge to run evaluations.

Explanation about CriteriaOption¶

A criteria option.