📄 Judge Context Relevance Logprobs¶

templates.rag_eval.context_relevance.judge_context_relevance_logprobs

type: InputOutputTemplateWithCustomTarget
input_format: "Question: {question}\n\nEvidence: {contexts}\n"
output_format: {is_context_relevant}
postprocessors: 
  - processors.load_json_from_predictions
  - processors.infer_logprobs_to_yes_no_probs
  - processors.cast_to_float_return_zero_if_failed
reference: {number_val}
target_prefix: Answer: 
instruction: "You are given a question and the corresponding evidence. Please determine whether the evidence contain the answer to the question. Answer with only yes/no.\n\n"
[source]

References: processors.cast_to_float_return_zero_if_failed, processors.infer_logprobs_to_yes_no_probs, processors.load_json_from_predictions