Unitxt: streamlining data processing

Unitxt is a Python library for getting data prepared and ready for utilization. In one line of code, it preps a dataset or mixtures-of-datasets into an input-output format for training and evaluation. Our aspiration is to be simple, adaptable, and transparent.

Contents:

• Introduction
• Loading Premade Dataset Recipes
  • Loading a Premade Dataset Recipe
  • Loading Customized Data
• Installation
• Concepts
  • Stream
  • Operator
  • Streaming
  • Artifact
  • Recipe
• Adding Datasets
  • The Task Card
  • Adding the Dataset to the Catalog
  • Sharing the Dataset
• Adding Stream Operators and Metrics
  • Adding a new stream operator
  • Adding a new metric
• Operators
• unitxt
  • unitxt package
• Catalog
  • recipes
  • templates
  • cards
  • instructions
  • formats
  • tasks
  • metrics
  • benchmarks
  • splitters
  • processors

Indices and tables

• Index

• Module Index

• Search Page