unitxt package

Subpackages

• unitxt.prepare_utils package
  • Submodules
    • unitxt.prepare_utils.card_types module
      • add_classification_choices()
      • create_2sentences_classification_card()
      • create_sentence_classification_card()
    • unitxt.prepare_utils.instructions module
  • Module contents
• unitxt.test_utils package
  • Submodules
    • unitxt.test_utils.artifact module
      • test_artfifact_saving_and_loading()
    • unitxt.test_utils.card module
      • construct_recipe_output_message()
      • load_examples_from_standard_recipe()
      • print_recipe_output()
      • test_adding_to_catalog()
      • test_card()
      • test_loading_from_catalog()
      • test_metrics_exist()
      • test_with_eval()
    • unitxt.test_utils.catalog module
      • register_local_catalog_for_tests()
    • unitxt.test_utils.metrics module
      • apply_metric()
      • dict_equal()
      • round_floats()
      • test_metric()
    • unitxt.test_utils.operators module
      • apply_operator()
      • check_operator()
      • check_operator_exception()
    • unitxt.test_utils.storage module
      • get_directory_size()
  • Module contents

Submodules

• unitxt.artifact module
  • Artifact
    • Artifact.from_dict()
    • Artifact.get_artifact_type()
    • Artifact.is_artifact_dict()
    • Artifact.is_artifact_file()
    • Artifact.is_registered_class()
    • Artifact.is_registered_class_name()
    • Artifact.is_registered_type()
    • Artifact.load()
    • Artifact.prepare()
    • Artifact.register_class()
    • Artifact.save()
    • Artifact.verify()
    • Artifact.verify_artifact_dict()
  • ArtifactList
  • Artifactories
    • Artifactories.instance
    • Artifactories.register()
    • Artifactories.reset()
    • Artifactories.unregister()
  • Artifactory
  • MissingArtifactTypeError
  • UnitxtArtifactNotFoundError
  • UnrecognizedArtifactTypeError
  • fetch_artifact()
  • get_closest_artifact_type()
  • map_values_in_place()
  • maybe_recover_artifact()
  • register_all_artifacts()
  • verbosed_fetch_artifact()
• unitxt.blocks module
• unitxt.card module
  • TaskCard
• unitxt.catalog module
  • Catalog
  • EnvironmentLocalCatalog
  • GithubCatalog
  • LocalCatalog
  • add_to_catalog()
  • verify_legal_catalog_name()
• unitxt.collections module
  • Collection
  • DictCollection
  • ItemPicker
  • ListCollection
  • RandomPicker
• unitxt.dataclass module
  • AbstractField
  • AbstractFieldError
  • Dataclass
    • Dataclass.to_dict()
  • DataclassMeta
  • Field
    • Field.default
    • Field.name
    • Field.type
    • Field.default_factory
    • Field.final
    • Field.abstract
    • Field.required
    • Field.origin_cls
    • Field.get_default()
  • FinalField
  • FinalFieldError
  • InternalField
  • NonPositionalField
  • OptionalField
  • RequiredField
  • RequiredFieldError
  • TypeMismatchError
  • UnexpectedArgumentError
  • abstract_fields()
  • asdict()
  • class_fields()
  • fields()
  • fields_names()
  • final_fields()
  • get_field_default()
  • get_fields()
  • is_abstract_field()
  • is_dataclass()
  • is_final_field()
  • is_possible_field()
  • required_fields()
• unitxt.dataset module
  • Dataset
    • Dataset.VERSION
    • Dataset.generators
  • fetch()
  • get_dataset_artifact()
  • parse()
• unitxt.dict_utils module
  • dict_creator()
  • dict_delete()
  • dict_get()
  • dict_set()
  • dpath_get()
  • dpath_set()
  • dpath_set_multiple()
  • dpath_set_one()
  • is_subpath()
• unitxt.file_utils module
  • get_all_files_in_dir()
• unitxt.formats module
  • Format
  • SystemFormat
• unitxt.fusion module
  • BaseFusion
  • FixedFusion
  • WeightedFusion
• unitxt.generator_utils module
  • CopyingReusableGenerator
  • ReusableGenerator
    • ReusableGenerator.activate()
• unitxt.hf_utils module
  • HFCachingContextManager
  • get_missing_imports()
• unitxt.instructions module
  • EmptyInstruction
  • Instruction
  • InstructionsDict
    • InstructionsDict.verify()
  • InstructionsList
  • TextualInstruction
• unitxt.load module
  • load_dataset()
• unitxt.loaders module
  • LoadCSV
  • LoadFromIBMCloud
  • LoadFromKaggle
  • LoadHF
  • Loader
  • MissingKaggleCredentialsError
• unitxt.logging_utils module
  • enable_explicit_format()
  • get_logger()
  • set_verbosity()
• unitxt.metric module
  • FromPredictionsAndOriginalData
  • Metric
  • MetricRecipe
  • MultiStreamScoreMean
• unitxt.metrics module
  • Accuracy
    • Accuracy.reduction_map
  • BertScore
    • BertScore.ci_scores
    • BertScore.hf_metric_fields
    • BertScore.reduction_map
  • BulkInstanceMetric
  • CharEditDistanceAccuracy
    • CharEditDistanceAccuracy.reduction_map
  • CustomF1
  • F1
  • F1Macro
  • F1MacroMultiLabel
  • F1Micro
  • F1MicroMultiLabel
  • F1MultiLabel
    • F1MultiLabel.classes_to_ignore
  • F1Weighted
  • GlobalMetric
  • HuggingfaceBulkMetric
    • HuggingfaceBulkMetric.hf_compute_args
  • HuggingfaceMetric
  • InstanceMetric
    • InstanceMetric.reduction_map
  • KPA
  • MAP
    • MAP.reduction_map
  • MRR
    • MRR.reduction_map
  • MatthewsCorrelation
  • Metric
    • Metric.main_score
  • MetricPipeline
  • MetricWithConfidenceInterval
  • NDCG
  • NER
  • PrecisionMacroMultiLabel
  • PrecisionMicroMultiLabel
  • RecallMacroMultiLabel
  • RecallMicroMultiLabel
  • RetrievalAtK
  • RetrievalMetric
  • Reward
    • Reward.reduction_map
  • Rouge
    • Rouge.rouge_types
  • SentenceBert
    • SentenceBert.reduction_map
  • Squad
  • StringContainment
    • StringContainment.reduction_map
  • TokenOverlap
    • TokenOverlap.ci_scores
    • TokenOverlap.reduction_map
  • UpdateStream
  • Wer
  • abstract_factory()
  • abstract_field()
  • normalize_answer()
• unitxt.normalizers module
  • NormalizeListFields
• unitxt.operator module
  • BaseFieldOperator
  • InstanceOperator
  • InstanceOperatorWithMultiStreamAccess
  • MultiStreamOperator
  • Operator
  • OperatorError
    • OperatorError.from_exception()
    • OperatorError.from_operator_error()
  • PagedStreamOperator
  • SequentialOperator
  • SequentialOperatorInitilizer
  • SingleStreamOperator
  • SingleStreamReducer
  • SourceOperator
  • SourceSequentialOperator
  • StreamInitializerOperator
  • StreamInstanceOperator
  • StreamInstanceOperatorValidator
  • StreamSource
  • StreamingOperator
• unitxt.operators module
  • Operators: Building Blocks of Unitxt Processing Pipelines
    • Creating Custom Operators
    • General or Specelized Operators
    • General Operaotrs List:
  • AddConstant
  • AddFields
  • AddID
  • Apply
  • ApplyMetric
  • ApplyOperatorsField
  • ApplyStreamOperatorsField
  • ArtifactFetcherMixin
    • ArtifactFetcherMixin.cache
    • ArtifactFetcherMixin.get_artifact()
  • AugmentPrefixSuffix
    • AugmentPrefixSuffix.prefixes
    • AugmentPrefixSuffix.suffixes
  • AugmentWhitespace
  • Augmentor
  • CastFields
  • CopyFields
  • DeterministicBalancer
    • DeterministicBalancer.fields
    • DeterministicBalancer.max_instances
  • DivideAllFieldsBy
  • EncodeLabels
  • ExecuteQuery
  • ExtractFieldValues
  • ExtractMostCommonFieldValues
  • FieldOperator
  • FilterByCondition
    • FilterByCondition.condition_to_func
  • FilterByQuery
  • FlattenInstances
  • FromIterables
  • IndexOf
  • Intersect
  • IterableSource
  • JoinStr
  • LengthBalancer
  • ListFieldValues
  • MapInstanceValues
    • MapInstanceValues.mappers
    • MapInstanceValues.strict
    • MapInstanceValues.process_every_value
  • MergeStreams
  • NullAugmentor
  • RemoveFields
  • RemoveValues
  • RenameFields
  • Shuffle
  • ShuffleFieldValues
  • SplitByValue
  • StreamRefiner
  • TakeByField
  • Unique
  • ZipFieldValues
• unitxt.processors module
  • ConvertToBoolean
  • DictOfListsToPairs
  • FirstCharacter
  • ListToEmptyEntitiesTuples
  • LoadJson
  • LowerCase
  • LowerCaseTillPunc
  • RegexParser
  • StanceToProCon
  • StringOrNotString
  • TakeFirstNonEmptyLine
  • TakeFirstWord
  • ToListByComma
  • ToString
  • ToStringStripped
  • ToYesOrNone
  • YesNoToInt
• unitxt.random_utils module
  • get_seed()
  • new_random_generator()
• unitxt.recipe module
  • Recipe
  • SequentialRecipe
• unitxt.register module
  • ProjectArtifactRegisterer
  • register_all_artifacts()
  • register_local_catalog()
• unitxt.schema module
  • ToUnitxtGroup
• unitxt.split_utils module
  • build_stream_routing()
  • parse_random_mix_string()
  • parse_slices_string()
  • random_mix_generator()
  • random_mix_streams()
  • rename_split()
  • slice_stream()
  • slice_streams()
• unitxt.splitters module
  • DiverseLabelsSampler
  • RandomSampler
  • RenameSplits
  • Sampler
    • Sampler.random_generator
  • SeparateSplit
  • SliceSplit
  • SplitRandomMix
  • Splitter
  • SpreadSplit
• unitxt.standard module
  • AddDemosField
  • BaseRecipe
  • CreateDemosPool
  • StandardRecipe
    • StandardRecipe.card
    • StandardRecipe.template
    • StandardRecipe.instruction
    • StandardRecipe.loader_limit
    • StandardRecipe.format
    • StandardRecipe.train_refiner
    • StandardRecipe.max_train_instances
    • StandardRecipe.validation_refiner
    • StandardRecipe.max_validation_instances
    • StandardRecipe.test_refiner
    • StandardRecipe.max_test_instances
    • StandardRecipe.demos_pool_size
    • StandardRecipe.num_demos
    • StandardRecipe.demos_pool_name
    • StandardRecipe.demos_taken_from
    • StandardRecipe.demos_field
    • StandardRecipe.sampler
    • StandardRecipe.steps
    • StandardRecipe.augmentor
    • StandardRecipe.instruction_card_index
    • StandardRecipe.template_card_index
    • StandardRecipe.prepare()
  • StandardRecipeWithIndexes
• unitxt.stream module
  • MultiStream
    • MultiStream.data
    • MultiStream.from_generators()
    • MultiStream.from_iterables()
    • MultiStream.get_generator()
    • MultiStream.set_caching()
    • MultiStream.set_copying()
    • MultiStream.to_dataset()
    • MultiStream.to_iterable_dataset()
  • Stream
    • Stream.generator
    • Stream.gen_kwargs
    • Stream.caching
    • Stream.peek()
    • Stream.take()
• unitxt.task module
  • FormTask
    • FormTask.augmentable_inputs
  • MultipleChoiceTask
  • Tasker
• unitxt.templates module
  • InputOutputTemplate
  • KeyValTemplate
  • MultiLabelTemplate
    • MultiLabelTemplate.postprocessors
  • MultiReferenceTemplate
  • MultipleChoiceTemplate
  • OutputQuantizingTemplate
  • SpanLabelingBaseTemplate
  • SpanLabelingJsonTemplate
    • SpanLabelingJsonTemplate.postprocessors
  • SpanLabelingTemplate
    • SpanLabelingTemplate.escape_characters
    • SpanLabelingTemplate.postprocessors
  • Template
  • TemplatesDict
    • TemplatesDict.verify()
  • TemplatesList
  • YesNoTemplate
  • escape_chars()
• unitxt.text_utils module
  • camel_to_snake_case()
  • construct_dict_str()
  • is_camel_case()
  • is_snake_case()
  • nested_tuple_to_string()
  • print_dict()
  • split_words()
• unitxt.type_utils module
  • NormalizedType
    • NormalizedType.args
    • NormalizedType.origin
  • eval_forward_ref()
  • get_args()
  • get_origin()
  • isoftype()
  • issubtype()
  • normalize()
  • optional_all()
  • optional_any()
• unitxt.utils module
  • Singleton
  • flatten_dict()
  • load_json()
  • save_json()
• unitxt.validate module
  • StandardSchema
  • ValidateSchema
  • ValidateStandartSchema
  • Validator
• unitxt.version module

Module contents