📄 Judge With Question Simplified Numeric¶

templates.rag_eval.faithfulness.judge_with_question_simplified_numeric

InputOutputTemplateWithCustomTarget(
    input_format="Question: {question}

Evidence: {contexts}

Prediction: {answer}
",
    output_format="{is_faithful}",
    postprocessors=[
        "processors.take_first_word",
        "processors.lower_case",
        "processors.cast_to_float_return_zero_if_failed",
        "processors.scale_0_10_to_0_1",
    ],
    reference="{number_val}",
    target_prefix="Answer: ",
    instruction="You are given a question, the corresponding evidence and a prediction from a model. Compare the "Prediction" and the "Evidence" to determine to what extent the prediction is grounded in the evidence.
To be grounded in the evidence, all the information of the prediction must either be present in the evidence or deducible from the evidence.

The question is only given for context, and is irrelevant for determining the groundedness of the prediction.
On a scale of 0 to 10, to what extent is the prediction grounded in the evidence? Reply with your rating score without any preceding explanation.

",
)
[source]

References: processors.cast_to_float_return_zero_if_failed, processors.scale_0_10_to_0_1, processors.take_first_word, processors.lower_case