📄 Judge Context Relevance Ares Verbal¶

templates.rag_eval.context_relevance.judge_context_relevance_ares_verbal

InputOutputTemplateWithCustomTarget(
    input_format="Question: {question}

Document: {contexts}
",
    output_format="{is_context_relevant}",
    postprocessors=[
        "processors.take_first_word",
        "processors.lower_case",
        "processors.extract_verbal_judgement",
    ],
    reference="{number_val}",
    target_prefix="Answer: ",
    instruction="Given the following question and document, you must analyze the provided document and determine to what extent it is sufficient for answering the question. In your evaluation, you should consider the content of the document and how it relates to the provided question.
Reply with one of the 4 options, without any further explanations:
"Completely Relevant" - if the question is completely answerable from the document.
"Mostly Relevant" - if the document contains most of the information required to answer the question but not enough to fully address it.
"Somewhat Relevant" - If the document is related to the question but not sufficient to answer it.
"Not Relevant" - If the document is irrelevant to the question.
",
)
[source]

References: processors.extract_verbal_judgement, processors.take_first_word, processors.lower_case