unitxt.standard module

class unitxt.standard.AddDemosField(*argv, **kwargs): Bases: SpreadSplit

class unitxt.standard.BaseRecipe(*argv, **kwargs): Bases: Recipe, SourceSequentialOperator

class unitxt.standard.CreateDemosPool(*argv, **kwargs): Bases: SeparateSplit

class unitxt.standard.StandardRecipe(*argv, **kwargs)

Bases: StandardRecipeWithIndexes

This class represents a standard recipe for data processing and preparation.

This class can be used to prepare a recipe. with all necessary steps, refiners and renderers included. It allows to set various parameters and steps in a sequential manner for preparing the recipe.

card

TaskCard object associated with the recipe.

Type:: TaskCard

template

Template object to be used for the recipe.

Type:: Template, optional

instruction

Instruction object to be used for the recipe.

Type:: Instruction, optional

loader_limit

Specifies the maximum number of instances per stream to be returned from the loader (used to reduce loading time in large datasets)

Type:: int, optional

format

SystemFormat object to be used for the recipe.

Type:: SystemFormat, optional

train_refiner

Train refiner to be used in the recipe.

Type:: StreamRefiner, optional

max_train_instances

Maximum training instances for the refiner.

Type:: int, optional

validation_refiner

Validation refiner to be used in the recipe.

Type:: StreamRefiner, optional

max_validation_instances

Maximum validation instances for the refiner.

Type:: int, optional

test_refiner

Test refiner to be used in the recipe.

Type:: StreamRefiner, optional

max_test_instances

Maximum test instances for the refiner.

Type:: int, optional

demos_pool_size

Size of the demos pool.

Type:: int, optional

num_demos

Number of demos to be used.

Type:: int, optional

demos_pool_name

Name of the demos pool. Default is “demos_pool”.

Type:: str, optional

demos_taken_from

Specifies from where the demos are taken. Default is “train”.

Type:: str, optional

demos_field

Field name for demos. Default is “demos”.

Type:: str, optional

sampler

Sampler object to be used in the recipe.

Type:: Sampler, optional

steps

List of StreamingOperator objects to be used in the recipe.

Type:: List[StreamingOperator], optional

augmentor

Augmentor to be used to pseudo randomly augment the source text

Type:: Augmentor

instruction_card_index

Index of instruction card to be used for preparing the recipe.

Type:: int, optional

template_card_index

Index of template card to be used for preparing the recipe.

Type:: int, optional

prepare(): This overridden method is used for preparing the recipe by arranging all the steps, refiners, and renderers in a sequential manner.

Raises:: AssertionError – If both template and template_card_index, or instruction and instruction_card_index are specified at the same time.

class unitxt.standard.StandardRecipeWithIndexes(*argv, **kwargs): Bases: BaseRecipe