📄 Judge No Question Simplified Explain Logprobs¶

Note

ID: templates.rag_eval.faithfulness.judge_no_question_simplified_explain_logprobs | Type: InputOutputTemplateWithCustomTarget

{
    "__type__": "input_output_template_with_custom_target",
    "input_format": "Evidence: {contexts}\n\nPrediction: {answer}\n",
    "instruction": "You are given an \"Evidence\" and a \"Prediction\" from a model. Compare the \"Prediction\" and the \"Evidence\" texts to determine whether all the information of the \"Prediction\" is present in the \"Evidence\" or can be inferred from the \"Evidence\". You must answer \"No\" if there are any specific details in the \"Prediction\" that are not mentioned in the \"Evidence\" or cannot be inferred from the \"Evidence\". Answer only \"Yes\" or \"No\". Then, provide an explanation to your answer.\n",
    "output_format": "{is_faithful}",
    "postprocessors": [
        "processors.load_json_from_predictions",
        "processors.infer_logprobs_to_yes_no_probs",
        "processors.cast_to_float_return_zero_if_failed"
    ],
    "reference": "{number_val}",
    "target_prefix": "Answer: "
}

References: processors.cast_to_float_return_zero_if_failed, processors.load_json_from_predictions, processors.infer_logprobs_to_yes_no_probs