📄 Lm Eval Harness¶

templates.qa.multiple_choice.with_context.with_topic.de.lm_eval_harness

type: MultipleChoiceTemplate
input_format: "Das folgende sind mehrfache auswahlfragen (mit antworten) bezueglich {topic}.\n\n{context}\n{question}.\n{choices}.\nAatwort:"
target_field: answer
choices_separator: "\n"
target_choice_format:  {choice_numeral}
postprocessors: 
  - processors.first_character
[source]

Explanation about MultipleChoiceTemplate¶

Formats the input (that specifies the question), the multiple choices to select the answer from, and specifies the field with the correct answer.

References: processors.first_character