📄 default¶

Note

ID: templates.evaluation.preference.default | Type: MultipleChoiceTemplate

{
    "choices_separator": "\n",
    "input_format": "Instruction:\n{instruction}\n{input_type}:\n{input}\nResponses:\n{choices}",
    "instruction": "For any Instruction and {input_type} given to a model, assess which {output_type} written by the model aligns most closely with the given instruction (choose from {numerals}).",
    "postprocessors": [
        "processors.to_string_stripped",
        "processors.first_character"
    ],
    "shuffle_choices": true,
    "target_field": "output_choice",
    "target_prefix": "{output_type}:\n",
    "title_fields": [
        "input_type",
        "output_type"
    ],
    "type": "multiple_choice_template"
}

Explanation about MultipleChoiceTemplate¶

Formats the input (that specifies the question), the multiple choices to select the answer from, and specifies the field with the correct answer.

References: processors.to_string_stripped, processors.first_character