📄 V3¶

templates.response_assessment.judges.topicality.v3

InputOutputTemplate(
    instruction="Read the following two parts: (1) The Conversation between the user and the agent, which can include multiple turns. The two speakers (user and agent) alternate in the conversation, and the user poses an inquiry at the end. (2) The Response (of the agent) to the last turn user query that continues the conversation. The response can rely on all the information provided in the previous turns. Your task is to evaluate if the Response (part 2) only contain relevant information to the inquiry, using one of three labels: [yes, no, unsure] ",
    input_format="

Conversation:
{question}

Response:
{answer}


Output:",
    output_format="[[{rating}]]",
    postprocessors=[
        "processors.take_first_word",
        "processors.lower_case",
        "processors.yes_no_to_int",
        "processors.cast_to_float_return_0_5_if_failed",
    ],
)
[source]

Explanation about InputOutputTemplate¶

Generate field ‘source’ from fields designated as input, and fields ‘target’ and ‘references’ from fields designated as output, of the processed instance.

Args specify the formatting strings with which to glue together the input and reference fields of the processed instance into one string (‘source’ and ‘target’), and into a list of strings (‘references’).

References: processors.cast_to_float_return_0_5_if_failed, processors.take_first_word, processors.yes_no_to_int, processors.lower_case