📄 Judge No Question Simplified Explain Logprobs¶

templates.rag_eval.faithfulness.judge_no_question_simplified_explain_logprobs

InputOutputTemplateWithCustomTarget(
    input_format="Evidence: {contexts}

Prediction: {answer}
",
    output_format="{is_faithful}",
    postprocessors=[
        "processors.load_json_from_predictions",
        "processors.infer_logprobs_to_yes_no_probs",
        "processors.cast_to_float_return_zero_if_failed",
    ],
    reference="{number_val}",
    target_prefix="Answer: ",
    instruction="You are given an "Evidence" and a "Prediction" from a model. Compare the "Prediction" and the "Evidence" texts to determine whether all the information of the "Prediction" is present in the "Evidence" or can be inferred from the "Evidence". You must answer "No" if there are any specific details in the "Prediction" that are not mentioned in the "Evidence" or cannot be inferred from the "Evidence". Answer only "Yes" or "No". Then, provide an explanation to your answer.
",
)
[source]

References: processors.cast_to_float_return_zero_if_failed, processors.infer_logprobs_to_yes_no_probs, processors.load_json_from_predictions