---

lm_eval_harness

Note

ID: catalog.templates.qa.multiple_choice.original.pt.lm_eval_harness | Type: MultipleChoiceTemplate

{
    "choices_seperator": "\n",
    "input_format": "A seguir estão perguntas de múltipla escolha (com respostas) sobre {topic}.\n\n{question}.\n{choices}\nResposta:",
    "postprocessors": [
        "processors.first_character"
    ],
    "target_choice_format": " {choice_numeral}",
    "target_field": "answer",
    "type": "multiple_choice_template"
}

Explanation about MultipleChoiceTemplate

Formats the input (that specifies the question), the multiple choices to select the answer from, and specifies the field with the correct answer.