📄 Judge Answer Relevance Numeric¶

templates.rag_eval.answer_relevance.judge_answer_relevance_numeric

type: InputOutputTemplateWithCustomTarget
input_format: "Question: {question}\n\nPrediction: {answer}\n"
output_format: {is_relevant}
postprocessors: 
  - processors.take_first_word
  - processors.lower_case
  - processors.cast_to_float_return_zero_if_failed
  - processors.scale_0_10_to_0_1
reference: {number_val}
target_prefix: Answer: 
instruction: "You are given a question and a prediction from a model. Please determine whether or not the prediction answers the question.\nThe prediction answers the question if it provides a relevant, appropriate and and complete answer to the question.\nIgnore the correctness of the prediction in your judgement.\nOn a scale of 0 to 10, to what extent does the prediction answer the question? Reply with your rating score without any preceding explanation.\n"
[source]

References: processors.cast_to_float_return_zero_if_failed, processors.scale_0_10_to_0_1, processors.take_first_word, processors.lower_case