Code Documentation¶
unitxt package¶
Subpackages¶
- unitxt.service package
- unitxt.test_utils package
- unitxt.ui package
Submodules¶
- unitxt.api module
- unitxt.artifact module
AbstractCatalog
Artifact
Artifact.from_dict()
Artifact.get_artifact_type()
Artifact.get_pretty_print_name()
Artifact.is_artifact_dict()
Artifact.is_artifact_file()
Artifact.is_registered_class()
Artifact.is_registered_class_name()
Artifact.is_registered_type()
Artifact.load()
Artifact.prepare()
Artifact.process_data_after_load()
Artifact.process_data_before_dump()
Artifact.register_class()
Artifact.save()
Artifact.serialize()
Artifact.to_json()
Artifact.verify()
Artifact.verify_artifact_dict()
Artifact.verify_data_classification_policy()
Artifact.verify_instance()
ArtifactList
Catalogs
MissingArtifactTypeError
UnitxtArtifactNotFoundError
UnrecognizedArtifactTypeError
fetch_artifact()
get_artifacts_data_classification()
get_catalog_name_and_args()
get_closest_artifact_type()
get_raw()
is_name_legal_for_catalog()
map_values_in_place()
maybe_recover_artifact()
register_all_artifacts()
reset_artifacts_json_cache()
verbosed_fetch_artifact()
verify_legal_catalog_name()
- unitxt.augmentors module
- unitxt.benchmark module
- unitxt.blocks module
- unitxt.card module
- unitxt.catalog module
- unitxt.collections module
- unitxt.collections_operators module
- unitxt.dataclass module
AbstractField
AbstractFieldError
Dataclass
DataclassMeta
Field
FinalField
FinalFieldError
InternalField
MissingDefaultError
NonPositionalField
OptionalField
RequiredField
RequiredFieldError
TypeMismatchError
Undefined
UnexpectedArgumentError
abstract_fields()
asdict()
class_fields()
fields()
fields_names()
final_fields()
get_field_default()
get_fields()
is_abstract_field()
is_class_method()
is_dataclass()
is_final_field()
is_possible_field()
required_fields()
- unitxt.dataset module
- unitxt.dataset_utils module
- unitxt.deprecation_utils module
- unitxt.dialog_operators module
- unitxt.dict_utils module
- unitxt.error_utils module
- unitxt.eval_utils module
- unitxt.file_utils module
- unitxt.formats module
- unitxt.fusion module
- unitxt.generator_utils module
- unitxt.hf_utils module
- unitxt.image_operators module
- unitxt.inference module
AsyncTokenBucket
CrossProviderInferenceEngine
GenericInferenceEngine
HFLlavaInferenceEngine
HFOptionSelectingInferenceEngine
HFPipelineBasedInferenceEngine
IbmGenAiInferenceEngine
IbmGenAiInferenceEngineParamsMixin
InferenceEngine
LMMSEvalBaseInferenceEngine
LMMSEvalInferenceEngine
LMMSEvalLoglikelihoodInferenceEngine
LazyLoadMixin
LiteLLMInferenceEngine
LogProbInferenceEngine
MockInferenceEngine
MockModeMixin
OllamaInferenceEngine
OpenAiInferenceEngine
OpenAiInferenceEngineParamsMixin
OptionSelectingByLogProbsInferenceEngine
StandardAPIParamsMixin
TextGenerationInferenceOutput
TogetherAiInferenceEngine
TogetherAiInferenceEngineParamsMixin
VLLMInferenceEngine
VLLMRemoteInferenceEngine
WMLInferenceEngine
WMLInferenceEngineParamsMixin
get_images_without_text()
get_model_and_label_id()
get_text_without_images()
- unitxt.instructions module
- unitxt.llm_as_judge module
LLMAsJudge
LLMAsJudgeBase
TaskBasedLLMasJudge
TaskBasedLLMasJudge.main_score
TaskBasedLLMasJudge.task
TaskBasedLLMasJudge.template
TaskBasedLLMasJudge.format
TaskBasedLLMasJudge.system_prompt
TaskBasedLLMasJudge.strip_system_prompt_and_format_from_inputs
TaskBasedLLMasJudge.inference_model
TaskBasedLLMasJudge.reduction_map
TaskBasedLLMasJudge.batch_size
TaskBasedLLMasJudge.infer_log_probs
TaskBasedLLMasJudge.judge_to_generator_fields_mapping
TaskBasedLLMasJudge.prediction_field
TaskBasedLLMasJudge.include_meta_data
TaskBasedLLMasJudge.judge_to_generator_fields_mapping
get_task_data_dict()
- unitxt.loaders module
- unitxt.logging_utils module
- unitxt.metric module
- unitxt.metric_utils module
DeleteTargetPrefix
FromPredictionsAndOriginalData
InstanceInput
JoinSubsetsAndGroups
MetricRecipe
MetricRequest
MetricResponse
PostProcessRecipe
SplitSubsetsAndGroups
get_remote_metrics_endpoint()
get_remote_metrics_names()
group_str()
group_str_to_key_value()
nan_mean()
stream_name_to_origin_subset_group()
- unitxt.metrics module
ANLS
Accuracy
BertScore
BinaryAccuracy
BinaryMaxAccuracy
BinaryMaxF1
BulkInstanceMetric
CharEditDistance
CharEditDistanceAccuracy
CustomF1
CustomF1Fuzzy
Detector
F1
F1Binary
F1BinaryPosOnly
F1Macro
F1MacroMultiLabel
F1Micro
F1MicroMultiLabel
F1MultiLabel
F1Strings
F1Weighted
FinQAEval
FixedGroupAbsvalNormCohensHParaphraseAccuracy
FixedGroupAbsvalNormCohensHParaphraseStringContainment
FixedGroupAbsvalNormHedgesGParaphraseAccuracy
FixedGroupAbsvalNormHedgesGParaphraseStringContainment
FixedGroupMeanAccuracy
FixedGroupMeanBaselineAccuracy
FixedGroupMeanBaselineStringContainment
FixedGroupMeanParaphraseAccuracy
FixedGroupMeanParaphraseStringContainment
FixedGroupMeanStringContainment
FixedGroupNormCohensHParaphraseAccuracy
FixedGroupNormCohensHParaphraseStringContainment
FixedGroupNormHedgesGParaphraseAccuracy
FixedGroupNormHedgesGParaphraseStringContainment
FixedGroupPDRParaphraseAccuracy
FixedGroupPDRParaphraseStringContainment
FuzzyNer
GlobalMetric
GroupMeanAccuracy
GroupMeanStringContainment
GroupMeanTokenOverlap
HuggingfaceBulkMetric
HuggingfaceInstanceMetric
HuggingfaceMetric
InstanceMetric
IsCodeMixed
JaccardIndex
KPA
KendallTauMetric
LlamaIndexCorrectness
LlamaIndexFaithfulness
LlamaIndexLLMMetric
MAP
MRR
MatthewsCorrelation
MaxAccuracy
Meteor
Metric
MetricPipeline
MetricWithConfidenceInterval
MetricsEnsemble
NDCG
NER
NLTKMixin
NormalizedSacrebleu
Perplexity
PrecisionBinary
PrecisionMacroMultiLabel
PrecisionMicroMultiLabel
PredictionLength
RandomForestMetricsEnsemble
RecallBinary
RecallMacroMultiLabel
RecallMicroMultiLabel
RegardMetric
RemoteMetric
RerankRecall
RetrievalAtK
RetrievalMetric
Reward
RocAuc
Rouge
RougeHF
SafetyMetric
SentenceBert
Spearmanr
Squad
StringContainment
StringContainmentRatio
TokenOverlap
UnsortedListExactMatch
UpdateStream
WeightedWinRateCorrelation
Wer
abstract_factory()
abstract_field()
interpret_effect_size()
mean_subgroup_score()
nan_max()
nan_mean()
normalize_answer()
normalized_cohens_h()
normalized_hedges_g()
parse_string_types_instead_of_actual_objects()
performance_drop_rate()
pytrec_eval_at_k()
validate_subgroup_types()
- unitxt.normalizers module
- unitxt.operator module
InstanceOperator
InstanceOperatorValidator
InstanceOperatorWithMultiStreamAccess
MissingRequirementsError
MultiStreamOperator
Operator
OperatorError
PackageRequirementsMixin
PagedStreamOperator
SequentialMixin
SequentialOperator
SequentialOperatorInitializer
SideEffectOperator
SingleStreamOperator
SingleStreamReducer
SourceOperator
SourceSequentialOperator
StreamInitializerOperator
StreamOperator
StreamingOperator
instance_generator()
instance_result()
stream_single()
- unitxt.operators module
- Operators: Building Blocks of Unitxt Processing Pipelines
AddConstant
AddID
Apply
ApplyMetric
ApplyOperatorsField
ApplyStreamOperatorsField
ArtifactFetcherMixin
Cast
CastFields
CollateInstances
ComputeExpressionMixin
Copy
DefaultPlaceHolder
DeterministicBalancer
DivideAllFieldsBy
DownloadError
DownloadOperator
DuplicateInstances
EncodeLabels
ExecuteExpression
ExtractFieldValues
ExtractMostCommonFieldValues
ExtractZipFile
FeatureGroupedShuffle
FieldOperator
FilterByCondition
FilterByConditionBasedOnFields
FilterByExpression
FlattenInstances
FromIterables
GetItemByIndex
IndexOf
InstanceFieldOperator
InterleaveListsToDialogOperator
Intersect
IterableSource
JoinStr
LengthBalancer
ListFieldValues
MapInstanceValues
MapValues
MergeStreams
MinimumOneExamplePerLabelRefiner
Perturb
RecursiveCopy
RemoveFields
RemoveValues
Rename
SelectFields
Set
Shuffle
ShuffleFieldValues
SplitByNestedGroup
SplitByValue
StreamRefiner
TakeByField
UnexpectedHttpCodeError
Unique
ZipFieldValues
- unitxt.parsing_utils module
- unitxt.processors module
Capitalize
ConvertToBoolean
DictOfListsToPairs
ExtractArenaHardNumericalJudgment
ExtractMtBenchLabelJudgment
ExtractMtBenchRatingJudgment
ExtractSafeUnsafeJudgment
ExtractWithRegex
FirstCharacter
FixWhiteSpace
GetStringAfter
InferDictsToBinaryLogprobs
ListToEmptyEntitiesTuples
LiteralEval
Lower
LowerCaseTillPunc
MatchClosestOption
PostProcess
RegexParser
RemoveArticles
RemovePunctuations
SplitStrip
StanceToProCon
StrToFloatFormat
StringEquals
Substring
TakeFirstNonEmptyLine
TakeFirstWord
ToListByComma
ToListByCommaSpace
ToString
ToStringStripped
ToYesOrNone
YesNoToInt
YesToOneElseZero
process_instance_value()
- unitxt.random_utils module
- unitxt.recipe module
- unitxt.register module
- unitxt.schema module
- unitxt.serializers module
- unitxt.settings_utils module
- unitxt.span_lableing_operators module
- unitxt.split_utils module
- unitxt.splitters module
- unitxt.standard module
BaseRecipe
CreateDemosPool
StandardRecipe
StandardRecipe.card
StandardRecipe.template
StandardRecipe.system_prompt
StandardRecipe.loader_limit
StandardRecipe.format
StandardRecipe.metrics
StandardRecipe.postprocessors
StandardRecipe.group_by
StandardRecipe.train_refiner
StandardRecipe.max_train_instances
StandardRecipe.validation_refiner
StandardRecipe.max_validation_instances
StandardRecipe.test_refiner
StandardRecipe.max_test_instances
StandardRecipe.demos_pool_size
StandardRecipe.num_demos
StandardRecipe.demos_pool_name
StandardRecipe.demos_taken_from
StandardRecipe.demos_field
StandardRecipe.demos_removed_from_data
StandardRecipe.sampler
StandardRecipe.steps
StandardRecipe.augmentor
StandardRecipe.instruction_card_index
StandardRecipe.template_card_index
StandardRecipe.prepare()
StandardRecipeWithIndexes
- unitxt.stream module
- unitxt.stream_operators module
- unitxt.string_operators module
- unitxt.struct_data_operators module
ConstructTableFromRowsCols
ConvertTableColNamesToSequential
DumpJson
DuplicateTableColumns
DuplicateTableRows
InsertEmptyTableRows
ListToKeyValPairs
LoadJson
MapHTMLTableToJSON
MapTableListsToStdTableJSON
MaskColumnsNames
SerializeKeyValPairs
SerializeTable
SerializeTableAsConcatenation
SerializeTableAsDFLoader
SerializeTableAsHTML
SerializeTableAsIndexedRowMajor
SerializeTableAsJson
SerializeTableAsMarkdown
SerializeTableRowAsList
SerializeTableRowAsText
SerializeTriples
ShuffleColumnsNames
ShuffleTableColumns
ShuffleTableRows
TransposeTable
TruncateTableCells
TruncateTableRows
shuffle_columns()
shuffle_rows()
truncate_cell()
- unitxt.system_prompts module
- unitxt.task module
- unitxt.templates module
ApplyRandomTemplate
ApplySingleTemplate
ApplyTemplate
DialogFieldsData
DialogPairwiseChoiceTemplate
DialogTemplate
InputFormatTemplate
InputOutputTemplate
InputOutputTemplateWithCustomTarget
KeyValTemplate
MultiLabelTemplate
MultiReferenceTemplate
MultipleChoiceTemplate
OutputFormatTemplate
OutputQuantizingTemplate
PairwiseChoiceTemplate
PairwiseComparativeRatingTemplate
SpanLabelingBaseTemplate
SpanLabelingJsonTemplate
SpanLabelingTemplate
Template
TemplateFormatKeyError
TemplatesDict
TemplatesList
YesNoTemplate
escape_chars()
random()
- unitxt.text_utils module
- unitxt.type_utils module
GenericTypedDict
NormalizedType
UnsupportedTypeError
convert_union_type()
eval_forward_ref()
format_type_string()
get_args()
get_origin()
infer_type()
infer_type_string()
is_new_type()
is_type()
is_type_dict()
is_typed_dict()
isoftype()
issubtype()
normalize()
optional_all()
optional_any()
parse_type_dict()
parse_type_string()
register_type()
replace_class_names()
strtype()
to_float_or_default()
to_type_dict()
to_type_string()
verify_required_schema()
- unitxt.types module
- unitxt.utils module
LRUCache
LongString
Singleton
artifacts_json_cache()
deep_copy()
flatten_dict()
import_module_from_file()
is_module_available()
is_package_installed()
json_dump()
load_json()
recursive_copy()
recursive_deep_copy()
recursive_shallow_copy()
remove_numerics_and_quoted_texts()
safe_eval()
save_to_file()
shallow_copy()
- unitxt.validate module
- unitxt.version module