📄 Judge Context Relevance Logprobs¶

Note

ID: templates.rag_eval.context_relevance.judge_context_relevance_logprobs | Type: InputOutputTemplateWithCustomTarget

{
    "__type__": "input_output_template_with_custom_target",
    "input_format": "Question: {question}\n\nEvidence: {contexts}\n",
    "instruction": "You are given a question and the corresponding evidence. Please determine whether the evidence contain the answer to the question. Answer with only yes/no.\n\n",
    "output_format": "{is_context_relevant}",
    "postprocessors": [
        "processors.load_json_from_predictions",
        "processors.infer_logprobs_to_yes_no_probs",
        "processors.cast_to_float_return_zero_if_failed"
    ],
    "reference": "{number_val}",
    "target_prefix": "Answer: "
}

References: processors.cast_to_float_return_zero_if_failed, processors.load_json_from_predictions, processors.infer_logprobs_to_yes_no_probs