📄 Flan T5 Small¶

metrics.perplexity_chat.flan_t5_small

Perplexity(
    model_name="google/flan-t5-small",
    source_template="Generate a conversation between a user and an agent based on the given content: {reference}",
    target_template="{prediction}",
)
[source]

Explanation about Perplexity¶

Computes perplexity of generating target text given source context.

Range: [1, ∞) (lower is better) Measures how well a language model predicts the target sequence.

Reference: https://en.wikipedia.org/wiki/Perplexity