📄 squad¶

Note

ID: metrics.squad | Type: MetricPipeline

{
    "main_score": "f1",
    "metric": {
        "type": "squad"
    },
    "preprocess_steps": [
        {
            "type": "add_id"
        },
        {
            "fields": {
                "prediction_template": {
                    "id": "ID",
                    "prediction_text": "PRED"
                },
                "reference_template": {
                    "answers": {
                        "answer_start": [
                            -1
                        ],
                        "text": "REF"
                    },
                    "id": "ID"
                }
            },
            "type": "add_fields",
            "use_deepcopy": true
        },
        {
            "field_to_field": [
                [
                    "references",
                    "reference_template/answers/text"
                ],
                [
                    "prediction",
                    "prediction_template/prediction_text"
                ],
                [
                    "id",
                    "prediction_template/id"
                ],
                [
                    "id",
                    "reference_template/id"
                ]
            ],
            "type": "copy_fields"
        },
        {
            "field_to_field": [
                [
                    "reference_template",
                    "references"
                ],
                [
                    "prediction_template",
                    "prediction"
                ]
            ],
            "type": "copy_fields"
        }
    ],
    "type": "metric_pipeline"
}

Explanation about AddID¶

Stores a unique id value in the designated ‘id_field_name’ field of the given instance.

Explanation about CopyFields¶

Copies values from specified fields to specified fields.

Args (of parent class):
field_to_field (Union[List[List], Dict[str, str]]): A list of lists, where each sublist contains the source field and the destination field, or a dictionary mapping source fields to destination fields.

Examples:
An input instance {“a”: 2, “b”: 3}, when processed by CopyField(field_to_field={“a”: “b”} would yield {“a”: 2, “b”: 2}, and when processed by CopyField(field_to_field={“a”: “c”} would yield {“a”: 2, “b”: 3, “c”: 2}

with field names containing / , we can also copy inside the field: CopyFields(field_to_field={“a/0”: “a”}) would process instance {“a”: [1, 3]} into {“a”: 1}

Explanation about AddFields¶

Adds specified fields to each instance in a given stream or all streams (default) If fields exist, updates them.

Args:

fields (Dict[str, object]): The fields to add to each instance.
Use ‘/’ to access inner fields

use_deepcopy (bool) : Deep copy the input value to avoid later modifications

Examples:
# Add a ‘classes’ field with a value of a list “positive” and “negative” to all streams AddFields(fields={“classes”: [“positive”,”negatives”]})

# Add a ‘start’ field under the ‘span’ field with a value of 0 to all streams AddFields(fields={“span/start”: 0}

# Add a ‘classes’ field with a value of a list “positive” and “negative” to ‘train’ stream AddFields(fields={“classes”: [“positive”,”negatives”], apply_to_stream=[“train”]})

# Add a ‘classes’ field on a given list, prevent modification of original list # from changing the instance. AddFields(fields={“classes”: alist}), use_deepcopy=True) # if now alist is modified, still the instances remain intact.