📄 Table2Text Single Turn With Reference¶

templates.response_assessment.rating.table2text_single_turn_with_reference

type: InputOutputTemplate
instruction: "Please act as an impartial judge and evaluate the quality of the text generated by an AI assistant to the table input(table, caption) given below. Your evaluation should consider correctness and helpfulness. You will be given a reference text and the assistant generated text. Begin your evaluation by comparing the assistant generated text with the reference text. Identify and correct any mistakes. Be as objective as possible. After providing your explanation, you must rate the generated text on a scale of 1 to 10 by strictly following this format: \"[[rating]]\", for example: \"Rating: [[5]]\".\n\n"
input_format: "[Input given to the AI assistant]\n{question}\n\n[The Start of Reference Text]\n{reference_answer}\n[The End of Reference Text]\n\n[The Start of Assistant's Generated Text]\n{answer}\n[The End of Assistant's Generated Text]"
output_format: [[{rating}]]
postprocessors: 
  - processors.extract_mt_bench_rating_judgment
[source]

Explanation about InputOutputTemplate¶

Generate field ‘source’ from fields designated as input, and fields ‘target’ and ‘references’ from fields designated as output, of the processed instance.

Args specify the formatting strings with which to glue together the input and reference fields of the processed instance into one string (‘source’ and ‘target’), and into a list of strings (‘references’).

References: processors.extract_mt_bench_rating_judgment