📄 End To End¶

Note

ID: tasks.rag.end_to_end | Type: Task

{
    "__type__": "task",
    "augmentable_inputs": [
        "question"
    ],
    "input_fields": {
        "metadata_field": "str",
        "question": "str",
        "question_id": "Any"
    },
    "metrics": [
        "metrics.rag.end_to_end.answer_correctness",
        "metrics.rag.end_to_end.answer_faithfulness",
        "metrics.rag.end_to_end.answer_reward",
        "metrics.rag.end_to_end.context_correctness",
        "metrics.rag.end_to_end.context_relevance"
    ],
    "prediction_type": "Dict[str, Any]",
    "reference_fields": {
        "is_answerable_label": "bool",
        "reference_answers": "List[str]",
        "reference_context_ids": "List[str]",
        "reference_contexts": "List[str]"
    }
}

Explanation about Task¶

Task packs the different instance fields into dictionaries by their roles in the task.

Attributes:

input_fields (Union[Dict[str, str], List[str]]):
Dictionary with string names of instance input fields and types of respective values. In case a list is passed, each type will be assumed to be Any.

reference_fields (Union[Dict[str, str], List[str]]):
Dictionary with string names of instance output fields and types of respective values. In case a list is passed, each type will be assumed to be Any.

metrics (List[str]): List of names of metrics to be used in the task. prediction_type (Optional[str]):

Need to be consistent with all used metrics. Defaults to None, which means that it will be set to Any.

defaults (Optional[Dict[str, Any]]):
An optional dictionary with default values for chosen input/output keys. Needs to be consistent with names and types provided in ‘input_fields’ and/or ‘output_fields’ arguments. Will not overwrite values if already provided in a given instance.

The output instance contains three fields:
“input_fields” whose value is a sub-dictionary of the input instance, consisting of all the fields listed in Arg ‘input_fields’. “reference_fields” – for the fields listed in Arg “reference_fields”. “metrics” – to contain the value of Arg ‘metrics’

References: metrics.rag.end_to_end.answer_reward, metrics.rag.end_to_end.context_relevance, metrics.rag.end_to_end.context_correctness, metrics.rag.end_to_end.answer_faithfulness, metrics.rag.end_to_end.answer_correctness