📄 Is Code Mixed¶
metrics.is_code_mixed
Explanation about IsCodeMixed¶
Uses a generative model to assess whether a given text is code-mixed.
Our goal is to identify whether a text is code-mixed, i.e., contains a mixture of different languages. The model is asked to identify the language of the text; if the model response begins with a number we take this as an indication that the text is code-mixed, for example: - Model response: “The text is written in 2 different languages” vs. - Model response: “The text is written in German”
Note that this metric is quite tailored to specific model-template combinations, as it relies on the assumption that the model will complete the answer prefix “The text is written in ___” in a particular way.
Read more about catalog usage here.