📄 Judge Simplified No Context¶

Note

ID: templates.rag_eval.answer_correctness.judge_simplified_no_context | Type: InputOutputTemplateWithCustomTarget

{
    "__type__": "input_output_template_with_custom_target",
    "input_format": "Question: {question}\n\nGround-truth answer: {ground_truths}\n\nPrediction: {answer}\n",
    "instruction": "You are given a question, the corresponding ground-truth answer and a prediction from a model. Compare the \"Ground-truth answer\" and the \"Prediction\" to determine whether the prediction is a correct and complete answer to the question. Answer with only yes/no.\n",
    "output_format": "{is_correct}",
    "postprocessors": [
        "processors.take_first_word",
        "processors.lower_case",
        "processors.yes_no_to_int",
        "processors.cast_to_float_return_zero_if_failed"
    ],
    "reference": "{number_val}",
    "target_prefix": "Answer: "
}

References: processors.cast_to_float_return_zero_if_failed, processors.take_first_word, processors.yes_no_to_int, processors.lower_case