unitxt.api module

unitxt.api.evaluate(predictions, data) → List[Dict[str, Any]]

unitxt.api.infer(instance_or_instances, engine: InferenceEngine, dataset_query: str | None = None, return_data: bool = False, return_log_probs: bool = False, return_meta_data: bool = False, **kwargs)

unitxt.api.load(source: SourceOperator | str)

unitxt.api.load_dataset(dataset_query: str | None = None, split: str | None = None, streaming: bool = False, disable_cache: bool | None = None, **kwargs) → DatasetDict | IterableDatasetDict | Dataset | IterableDataset

Loads dataset.

If the ‘dataset_query’ argument is provided, then dataset is loaded from a card in local catalog based on parameters specified in the query.

Alternatively, dataset is loaded from a provided card based on explicitly given parameters.

Parameters:

• dataset_query (str, optional) – A string query which specifies a dataset to load from local catalog or name of specific recipe or benchmark in the catalog.

• example (For) – "card=cards.wnli,template=templates.classification.multi_class.relation.default".

• streaming (bool, False) – When True yields the data as Unitxt streams dictionary

• split (str, optional) – The split of the data to load

• disable_cache (str, optional) – Disable caching process of the data

• **kwargs – Arguments used to load dataset from provided card, which is not present in local catalog.

Returns:

DatasetDict

Example

dataset = load_dataset(
    dataset_query="card=cards.stsb,template=templates.regression.two_texts.simple,max_train_instances=5"
)  # card must be present in local catalog

card = TaskCard(...)
template = Template(...)
loader_limit = 10
dataset = load_dataset(card=card, template=template, loader_limit=loader_limit)

unitxt.api.load_recipe(dataset_query: str | None = None, **kwargs) → StandardRecipe

unitxt.api.post_process(predictions, data) → List[Dict[str, Any]]

unitxt.api.produce(instance_or_instances, dataset_query: str | None = None, **kwargs) → Dataset | Dict[str, Any]