📄 Judge Answer Relevance Numeric¶

templates.rag_eval.answer_relevance.judge_answer_relevance_numeric

InputOutputTemplateWithCustomTarget(
    input_format="Question: {question}

Prediction: {answer}
",
    output_format="{is_relevant}",
    postprocessors=[
        "processors.take_first_word",
        "processors.lower_case",
        "processors.cast_to_float_return_zero_if_failed",
        "processors.scale_0_10_to_0_1",
    ],
    reference="{number_val}",
    target_prefix="Answer: ",
    instruction="You are given a question and a prediction from a model. Please determine whether or not the prediction answers the question.
The prediction answers the question if it provides a relevant, appropriate and and complete answer to the question.
Ignore the correctness of the prediction in your judgement.
On a scale of 0 to 10, to what extent does the prediction answer the question? Reply with your rating score without any preceding explanation.
",
)
[source]

References: processors.cast_to_float_return_zero_if_failed, processors.scale_0_10_to_0_1, processors.take_first_word, processors.lower_case