HumanEvaluationCustomMetric¶
Structure Class¶
dataclass
¶
In a model evaluation job that uses human workers you must define the
name of the metric, and how you want that metric rated ratingMethod,
and an optional description of the metric.
Attributes¶
class-attribute
instance-attribute
¶
description: str | None = field(repr=False, default=None)
An optional description of the metric. Use this parameter to provide more details about the metric.
class-attribute
instance-attribute
¶
name: str = field(repr=False)
The name of the metric. Your human evaluators will see this name in the evaluation UI.
instance-attribute
¶
rating_method: str
Choose how you want your human workers to evaluation your model. Valid
values for rating methods are ThumbsUpDown,
IndividualLikertScale,ComparisonLikertScale, ComparisonChoice, and
ComparisonRank