📄 With Type¶

This is the Question Answering Task with provided context (which is a either text, image, audio, table , or dialog) and additional field called type. The ‘tasks.qa.open’ should be used if there is no context. One or more ground truth answers can be provided in the ‘answers’ field. By default, classical Rouge metric is used , but list of additional applicable metrics can be found under ‘metrics.qa’ in the Unitxt catalog.

tasks.qa.with_context.with_type

Task(
    input_fields={
        "context": "Union[Text, Image, Audio, Table, Dialog]",
        "context_type": "str",
        "question": "str",
    },
    reference_fields={
        "answers": "List[str]",
        "type": "str",
    },
    prediction_type="str",
    metrics=[
        "metrics.rouge",
    ],
    augmentable_inputs=[
        "context",
        "question",
    ],
    defaults={
        "answers": [],
    },
    default_template="templates.qa.with_context",
)
[source]

Explanation about Task¶

Task packs the different instance fields into dictionaries by their roles in the task.

Args:

input_fields (Union[Dict[str, str], List[str]]):
Dictionary with string names of instance input fields and types of respective values. In case a list is passed, each type will be assumed to be Any.

reference_fields (Union[Dict[str, str], List[str]]):
Dictionary with string names of instance output fields and types of respective values. In case a list is passed, each type will be assumed to be Any.

metrics (List[str]):
List of names of metrics to be used in the task.

prediction_type (Optional[str]):
Need to be consistent with all used metrics. Defaults to None, which means that it will be set to Any.

defaults (Optional[Dict[str, Any]]):
An optional dictionary with default values for chosen input/output keys. Needs to be consistent with names and types provided in ‘input_fields’ and/or ‘output_fields’ arguments. Will not overwrite values if already provided in a given instance.

The output instance contains three fields:

“input_fields” whose value is a sub-dictionary of the input instance, consisting of all the fields listed in Arg ‘input_fields’.

“reference_fields” – for the fields listed in Arg “reference_fields”.

“metrics” – to contain the value of Arg ‘metrics’

References: templates.qa.with_context, metrics.rouge