📄 Judge Context Relevance Ares Verbal Good Bad¶

templates.rag_eval.context_relevance.judge_context_relevance_ares_verbal_good_bad

InputOutputTemplateWithCustomTarget(
    input_format="Question: {question}

Document: {contexts}
",
    output_format="{is_context_relevant}",
    postprocessors=[
        "processors.extract_from_double_brackets",
        "processors.extract_verbal_judgement_bad_good",
        "processors.cast_to_float_return_zero_if_failed",
    ],
    reference="{number_val}",
    target_prefix="Answer: ",
    instruction="Given the following question and document, you must analyze the provided document and determine to what extent it is sufficient for answering the question. In your evaluation, you should consider the content of the document and how it relates to the provided question.
Provide a rating from one of the following choices: 'Very Bad', 'Bad', 'Mediocre', 'Good', 'Very Good'. Reply using the format of [[rating]], for example: '[[Mediocre]]'.
",
)
[source]

References: processors.cast_to_float_return_zero_if_failed, processors.extract_verbal_judgement_bad_good, processors.extract_from_double_brackets