unitxt.metric_utils module¶

class unitxt.metric_utils.FromPredictionsAndOriginalData¶: Bases: StreamInitializerOperator

class unitxt.metric_utils.InstanceInput(references: List[Any], additional_inputs: Dict | None = None)¶

A single instance inputted to a metric service.

class unitxt.metric_utils.MetricRecipe(steps: List[StreamingOperator], calc_confidence_intervals: bool = True)¶: Bases: SequentialOperatorInitilizer

class unitxt.metric_utils.MetricRequest¶

Bases: Dataclass

A request to a metrics service, includes a list of input instances.

class unitxt.metric_utils.MetricResponse(global_score: Dict[str, Any])¶

A response produced by a metrics service, includes the computed scores.

class unitxt.metric_utils.MultiStreamScoreMean¶: Bases: MultiStreamOperator

unitxt.metric_utils.get_remote_metrics_endpoint() → str¶

Load the remote metrics endpoint from an environment variable.

Returns:: str - The remote endpoint on which the remote metrics are available.

unitxt.metric_utils.get_remote_metrics_names() → List[str]¶

Load the remote metrics names from an environment variable.