📄 Metric Based Ner¶

Calculates f1 metrics for NER , by comparing entity using a provided Unitxt metric.

This customizable metric can use any Unitxt metric to compare entities, including LLM as Judge. The metric must accept string prediction and references as input. The similarity threshold is set by the ‘min_score_for_match’ attribute.

By default it uses accuracy (exact match), but the metic can be overridden.

metrics.metric_based_ner[metric=metrics.llm_as_judge.direct.watsonx.llama3_3_70b[criteria=metrics.llm_as_judge.direct.criteria.correctness_based_on_ground_truth,context_fields=ground_truth]]

metrics.metric_based_ner

MetricBasedNer(
    metric="metrics.accuracy",
    n_resamples=0,
    min_score_for_match=0.75,
)
[source]

Explanation about MetricBasedNer¶

Calculates f1 metrics for NER , by comparing entity using a provided Unitxt metric.

While the Ner metric uses exact match to compare entities and FuzzyNer uses fuzzy matching, this customiziable metric can use any Unitxt metric to compare entities, including LLM as Judge. The metric must acceptstring prediction and references as input. The similarity threshold is set by the ‘min_score_for_match’ attribute.

Example: MetricBasedNer(metric=Rouge(), min_score_for_match=0.9)

MetricBasedNer(metric=”metrics.llm_as_judge.direct.watsonx.llama3_3_70b[criteria=metrics.llm_as_judge.direct.criteria.correctness_based_on_ground_truth,context_fields=ground_truth]”)

References: metrics.accuracy