📄 Relation

This is a special case of multi class text classification task, in which we classify the relation between two texts. For example, whether one text entails another. The inputs are provided in “text_a” and “text_a” The set of ‘classes’ is a list of option of the relationship (e.g. “entailment”, “contradiction”, “neutral”) The ‘text_a_type’ and ‘text_type” are optional fields that defines the type of text we classify (e.g. “document”, “review”, etc.). The ‘type_of_relation’ is a required field that the defines the type of relation we identify (e.g. “entailment”) The ‘text_a_type’,’text_b_type’ and ‘type_of_relation’ fields can be used by the template to customize the prompt.

The default reported metrics are the classical f1_micro (equivalent to accuracy for multi class classification), and f1_macro.

tasks.classification.multi_class.relation

type: Task
input_fields: 
  text_a: Union[Text, Image, Audio, Table, Dialog]
  text_a_type: str
  text_b: str
  text_b_type: str
  classes: List[str]
  type_of_relation: str
reference_fields: 
  label: str
prediction_type: str
metrics: 
  - metrics.f1_micro
  - metrics.accuracy
  - metrics.f1_macro
augmentable_inputs: 
  - text_a
  - text_b
defaults: 
  text_a_type: first text
  text_b_type: second text
default_template: templates.classification.multi_class.relation.default
[source]

Explanation about Task

Task packs the different instance fields into dictionaries by their roles in the task.

Args:
input_fields (Union[Dict[str, str], List[str]]):

Dictionary with string names of instance input fields and types of respective values. In case a list is passed, each type will be assumed to be Any.

reference_fields (Union[Dict[str, str], List[str]]):

Dictionary with string names of instance output fields and types of respective values. In case a list is passed, each type will be assumed to be Any.

metrics (List[str]):

List of names of metrics to be used in the task.

prediction_type (Optional[str]):

Need to be consistent with all used metrics. Defaults to None, which means that it will be set to Any.

defaults (Optional[Dict[str, Any]]):

An optional dictionary with default values for chosen input/output keys. Needs to be consistent with names and types provided in ‘input_fields’ and/or ‘output_fields’ arguments. Will not overwrite values if already provided in a given instance.

The output instance contains three fields:
  1. “input_fields” whose value is a sub-dictionary of the input instance, consisting of all the fields listed in Arg ‘input_fields’.

  2. “reference_fields” – for the fields listed in Arg “reference_fields”.

  3. “metrics” – to contain the value of Arg ‘metrics’

References: templates.classification.multi_class.relation.default, metrics.f1_micro, metrics.accuracy, metrics.f1_macro

Read more about catalog usage here.