📄 Judge Context Relevance Logprobs¶

templates.rag_eval.context_relevance.judge_context_relevance_logprobs

InputOutputTemplateWithCustomTarget(
    input_format="Question: {question}

Evidence: {contexts}
",
    output_format="{is_context_relevant}",
    postprocessors=[
        "processors.load_json_from_predictions",
        "processors.infer_logprobs_to_yes_no_probs",
        "processors.cast_to_float_return_zero_if_failed",
    ],
    reference="{number_val}",
    target_prefix="Answer: ",
    instruction="You are given a question and the corresponding evidence. Please determine whether the evidence contain the answer to the question. Answer with only yes/no.

",
)
[source]

References: processors.cast_to_float_return_zero_if_failed, processors.infer_logprobs_to_yes_no_probs, processors.load_json_from_predictions