マニフェストファイルを使用したデータセットの作成 (SDK）

データセットを作成するには (SDK）

1. まだインストールしていない場合は、 と をインストール AWS CLI して設定します AWS SDKs。詳細については、「ステップ 4: AWS CLI と をセットアップする AWS SDKs」を参照してください。

2. 次のサンプルコードを使用して、データセットを作成します。

   CLI

   以下の値を変更します:

   • project-name をデータセットと関連付けるプロジェクトの名前に。

   • dataset-type を作成したいデータセットのタイプ (train または test) に。

   • dataset-source をマニフェストファイルの Amazon S3 の保存場所に設定します。

   • Bucket をマニフェストファイルを含む Amazon S3 バケットの名前に。

   • Key を Amazon S3 バケット内のマニフェストファイルのパスとファイル名に。

   aws lookoutvision create-dataset --project-name project\
     --dataset-type train or test\
     --dataset-source '{ "GroundTruthManifest": { "S3Object": { "Bucket": "bucket", "Key": "manifest file" } } }' \
     --profile lookoutvision-access
   

   Python

   このコードは、 AWS ドキュメントSDKサンプル GitHub リポジトリから取得されます。詳しい事例はこちらです。

       @staticmethod
       def create_dataset(lookoutvision_client, project_name, manifest_file, dataset_type):
           """
           Creates a new Lookout for Vision dataset
   
           :param lookoutvision_client: A Lookout for Vision Boto3 client.
           :param project_name: The name of the project in which you want to
                                create a dataset.
           :param bucket: The bucket that contains the manifest file.
           :param manifest_file: The path and name of the manifest file.
           :param dataset_type: The type of the dataset (train or test).
           """
           try:
               bucket, key = manifest_file.replace("s3://", "").split("/", 1)
               logger.info("Creating %s dataset type...", dataset_type)
               dataset = {
                   "GroundTruthManifest": {"S3Object": {"Bucket": bucket, "Key": key}}
               }
               response = lookoutvision_client.create_dataset(
                   ProjectName=project_name,
                   DatasetType=dataset_type,
                   DatasetSource=dataset,
               )
               logger.info("Dataset Status: %s", response["DatasetMetadata"]["Status"])
               logger.info(
                   "Dataset Status Message: %s",
                   response["DatasetMetadata"]["StatusMessage"],
               )
               logger.info("Dataset Type: %s", response["DatasetMetadata"]["DatasetType"])
   
               # Wait until either created or failed.
               finished = False
               status = ""
               dataset_description = {}
               while finished is False:
                   dataset_description = lookoutvision_client.describe_dataset(
                       ProjectName=project_name, DatasetType=dataset_type
                   )
                   status = dataset_description["DatasetDescription"]["Status"]
   
                   if status == "CREATE_IN_PROGRESS":
                       logger.info("Dataset creation in progress...")
                       time.sleep(2)
                   elif status == "CREATE_COMPLETE":
                       logger.info("Dataset created.")
                       finished = True
                   else:
                       logger.info(
                           "Dataset creation failed: %s",
                           dataset_description["DatasetDescription"]["StatusMessage"],
                       )
                       finished = True
   
               if status != "CREATE_COMPLETE":
                   message = dataset_description["DatasetDescription"]["StatusMessage"]
                   logger.exception("Couldn't create dataset: %s", message)
                   raise Exception(f"Couldn't create dataset: {message}")
   
           except ClientError:
               logger.exception("Service error: Couldn't create dataset.")
               raise
   
   

   Java V2

   このコードは、 AWS ドキュメントSDKサンプル GitHub リポジトリから取得されます。詳しい例はこちらで参照できます。

   /**
    * Creates an Amazon Lookout for Vision dataset from a manifest file.
    * Returns after Lookout for Vision creates the dataset.
    * 
    * @param lfvClient    An Amazon Lookout for Vision client.
    * @param projectName  The name of the project in which you want to create a
    *                     dataset.
    * @param datasetType  The type of dataset that you want to create (train or
    *                     test).
    * @param bucket       The S3 bucket that contains the manifest file.
    * @param manifestFile The name and location of the manifest file within the S3
    *                     bucket.
    * @return DatasetDescription The description of the created dataset.
    */
   public static DatasetDescription createDataset(LookoutVisionClient lfvClient,
                   String projectName,
                   String datasetType,
                   String bucket,
                   String manifestFile)
                   throws LookoutVisionException, InterruptedException {
   
           logger.log(Level.INFO, "Creating {0} dataset for project {1}",
                           new Object[] { projectName, datasetType });
   
           // Build the request. If no bucket supplied, setup for empty dataset creation.
           CreateDatasetRequest createDatasetRequest = null;
   
           if (bucket != null && manifestFile != null) {
   
                   InputS3Object s3Object = InputS3Object.builder()
                                   .bucket(bucket)
                                   .key(manifestFile)
                                   .build();
   
                   DatasetGroundTruthManifest groundTruthManifest = DatasetGroundTruthManifest.builder()
                                   .s3Object(s3Object)
                                   .build();
   
                   DatasetSource datasetSource = DatasetSource.builder()
                                   .groundTruthManifest(groundTruthManifest)
                                   .build();
   
                   createDatasetRequest = CreateDatasetRequest.builder()
                                   .projectName(projectName)
                                   .datasetType(datasetType)
                                   .datasetSource(datasetSource)
                                   .build();
           } else {
                   createDatasetRequest = CreateDatasetRequest.builder()
                                   .projectName(projectName)
                                   .datasetType(datasetType)
                                   .build();
           }
   
           lfvClient.createDataset(createDatasetRequest);
   
           DatasetDescription datasetDescription = null;
   
           boolean finished = false;
   
           // Wait until dataset is created, or failure occurs.
           while (!finished) {
   
                   datasetDescription = describeDataset(lfvClient, projectName, datasetType);
   
                   switch (datasetDescription.status()) {
                           case CREATE_COMPLETE:
                                   logger.log(Level.INFO, "{0}dataset created for project {1}",
                                                   new Object[] { datasetType, projectName });
                                   finished = true;
                                   break;
                           case CREATE_IN_PROGRESS:
                                   logger.log(Level.INFO, "{0} dataset creating for project {1}",
                                                   new Object[] { datasetType, projectName });
   
                                   TimeUnit.SECONDS.sleep(5);
   
                                   break;
   
                           case CREATE_FAILED:
                                   logger.log(Level.SEVERE,
                                                   "{0} dataset creation failed for project {1}. Error {2}",
                                                   new Object[] { datasetType, projectName,
                                                                   datasetDescription.statusAsString() });
                                   finished = true;
                                   break;
                           default:
                                   logger.log(Level.SEVERE, "{0} error when creating {1} dataset for project {2}",
                                                   new Object[] { datasetType, projectName,
                                                                   datasetDescription.statusAsString() });
                                   finished = true;
                                   break;
   
                   }
           }
   
           logger.log(Level.INFO, "Dataset info. Status: {0}\n Message: {1} }",
                           new Object[] { datasetDescription.statusAsString(),
                                           datasetDescription.statusMessage() });
   
           return datasetDescription;
   
   }

3. モデルのトレーニング (SDK） の手順に従ってモデルをトレーニングします。