EvaluationDataset¶
Structure Class¶
dataclass
¶
Used to specify the name of a built-in prompt dataset and optionally, the Amazon S3 bucket where a custom prompt dataset is saved.
Attributes¶
class-attribute
instance-attribute
¶
dataset_location: EvaluationDatasetLocation | None = None
For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved.
class-attribute
instance-attribute
¶
name: str = field(repr=False)
Used to specify supported built-in prompt datasets. Valid values are
Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions,
Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA,
Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and
Builtin.Wikitext2.