Doc AWS SDK 예제 GitHub 리포지토리에서 더 많은 SDK 예제를 사용할 수 있습니다. [AWS](https://github.com/awsdocs/aws-doc-sdk-examples)
기계 번역으로 제공되는 번역입니다. 제공된 번역과 원본 영어의 내용이 상충하는 경우에는 영어 버전이 우선합니다.
# SDK for Kotlin을 사용한 SageMaker AI 예제
다음 코드 예제에서는 AWS SDK for Kotlin을 SageMaker AI와 함께 사용하여 작업을 수행하고 일반적인 시나리오를 구현하는 방법을 보여줍니다.
*작업*은 대규모 프로그램에서 발췌한 코드이며 컨텍스트에 맞춰 실행해야 합니다. 작업은 개별 서비스 함수를 직접 호출하는 방법을 보여주며, 관련 시나리오의 컨텍스트에 맞는 작업을 볼 수 있습니다.
*시나리오*는 동일한 서비스 내에서 또는 다른 AWS 서비스와 결합된 상태에서 여러 함수를 직접적으로 호출하여 특정 태스크를 수행하는 방법을 보여주는 코드 예제입니다.
각 예시에는 전체 소스 코드에 대한 링크가 포함되어 있으며, 여기에서 컨텍스트에 맞춰 코드를 설정하고 실행하는 방법에 대한 지침을 찾을 수 있습니다.
**Topics**
+ [시작하기](#get_started)
+ [작업](#actions)
+ [시나리오](#scenarios)
## 시작하기
### SageMaker AI 시작
다음 코드 예제에서는 SageMaker AI 사용을 시작하는 방법을 보여줍니다.
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예제 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/services/sagemaker#code-examples)에서 전체 예제를 찾고 설정 및 실행하는 방법을 배워보세요.
```
suspend fun listBooks() {
SageMakerClient.fromEnvironment { region = "us-west-2" }.use { sageMakerClient ->
val response = sageMakerClient.listNotebookInstances(ListNotebookInstancesRequest {})
response.notebookInstances?.forEach { item ->
println("The notebook name is: ${item.notebookInstanceName}")
}
}
}
```
+ API 세부 정보는 *AWS SDK for Kotlin API 참조*의 [ListNotebookInstances](https://sdk.amazonaws.com/kotlin/api/latest/index.html)를 참조하세요.
## 작업
### `CreatePipeline`
다음 코드 예시는 `CreatePipeline`의 사용 방법을 보여줍니다.
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/usecases/workflow_sagemaker_pipes#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
// Create a pipeline from the example pipeline JSON.
suspend fun setupPipeline(filePath: String?, roleArnVal: String?, functionArnVal: String?, pipelineNameVal: String?) {
println("Setting up the pipeline.")
val parser = JSONParser()
// Read JSON and get pipeline definition.
FileReader(filePath).use { reader ->
val obj: Any = parser.parse(reader)
val jsonObject: JSONObject = obj as JSONObject
val stepsArray: JSONArray = jsonObject.get("Steps") as JSONArray
for (stepObj in stepsArray) {
val step: JSONObject = stepObj as JSONObject
if (step.containsKey("FunctionArn")) {
step.put("FunctionArn", functionArnVal)
}
}
println(jsonObject)
// Create the pipeline.
val pipelineRequest = CreatePipelineRequest {
pipelineDescription = "Kotlin SDK example pipeline"
roleArn = roleArnVal
pipelineName = pipelineNameVal
pipelineDefinition = jsonObject.toString()
}
SageMakerClient { region = "us-west-2" }.use { sageMakerClient ->
sageMakerClient.createPipeline(pipelineRequest)
}
}
}
```
+ API 세부 정보는 *AWS SDK for Kotlin API 참조*의 [CreatePipeline](https://sdk.amazonaws.com/kotlin/api/latest/index.html)을 참조하세요.
### `DeletePipeline`
다음 코드 예시는 `DeletePipeline`의 사용 방법을 보여줍니다.
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예제 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/usecases/workflow_sagemaker_pipes#code-examples)에서 전체 예제를 찾고 설정 및 실행하는 방법을 배워보세요.
```
// Delete a SageMaker pipeline by name.
suspend fun deletePipeline(pipelineNameVal: String) {
val pipelineRequest = DeletePipelineRequest {
pipelineName = pipelineNameVal
}
SageMakerClient { region = "us-west-2" }.use { sageMakerClient ->
sageMakerClient.deletePipeline(pipelineRequest)
println("*** Successfully deleted $pipelineNameVal")
}
}
```
+ API 세부 정보는 *AWS SDK for Kotlin API 참조*의 [DeletePipeline](https://sdk.amazonaws.com/kotlin/api/latest/index.html)을 참조하세요.
### `DescribePipelineExecution`
다음 코드 예시는 `DescribePipelineExecution`의 사용 방법을 보여줍니다.
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예제 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/usecases/workflow_sagemaker_pipes#code-examples)에서 전체 예제를 찾고 설정 및 실행하는 방법을 배워보세요.
```
suspend fun waitForPipelineExecution(executionArn: String?) {
var status: String
var index = 0
do {
val pipelineExecutionRequest = DescribePipelineExecutionRequest {
pipelineExecutionArn = executionArn
}
SageMakerClient { region = "us-west-2" }.use { sageMakerClient ->
val response = sageMakerClient.describePipelineExecution(pipelineExecutionRequest)
status = response.pipelineExecutionStatus.toString()
println("$index. The status of the pipeline is $status")
TimeUnit.SECONDS.sleep(4)
index++
}
} while ("Executing" == status)
println("Pipeline finished with status $status")
}
```
+ API 세부 정보는 *AWS SDK for Kotlin API 참조*의 [DescribePipelineExecution](https://sdk.amazonaws.com/kotlin/api/latest/index.html)을 참조하세요.
### `StartPipelineExecution`
다음 코드 예시는 `StartPipelineExecution`의 사용 방법을 보여줍니다.
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예제 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/usecases/workflow_sagemaker_pipes#code-examples)에서 전체 예제를 찾고 설정 및 실행하는 방법을 배워보세요.
```
// Start a pipeline run with job configurations.
suspend fun executePipeline(bucketName: String, queueUrl: String?, roleArn: String?, pipelineNameVal: String): String? {
println("Starting pipeline execution.")
val inputBucketLocation = "s3://$bucketName/samplefiles/latlongtest.csv"
val output = "s3://$bucketName/outputfiles/"
val gson = GsonBuilder()
.setFieldNamingPolicy(FieldNamingPolicy.UPPER_CAMEL_CASE)
.setPrettyPrinting()
.create()
// Set up all parameters required to start the pipeline.
val parameters: MutableList = java.util.ArrayList()
val para1 = Parameter {
name = "parameter_execution_role"
value = roleArn
}
val para2 = Parameter {
name = "parameter_queue_url"
value = queueUrl
}
val inputJSON = """{
"DataSourceConfig": {
"S3Data": {
"S3Uri": "s3://$bucketName/samplefiles/latlongtest.csv"
},
"Type": "S3_DATA"
},
"DocumentType": "CSV"
}"""
println(inputJSON)
val para3 = Parameter {
name = "parameter_vej_input_config"
value = inputJSON
}
// Create an ExportVectorEnrichmentJobOutputConfig object.
val jobS3Data = VectorEnrichmentJobS3Data {
s3Uri = output
}
val outputConfig = ExportVectorEnrichmentJobOutputConfig {
s3Data = jobS3Data
}
val gson4: String = gson.toJson(outputConfig)
val para4: Parameter = Parameter {
name = "parameter_vej_export_config"
value = gson4
}
println("parameter_vej_export_config:" + gson.toJson(outputConfig))
val para5JSON =
"{\"MapMatchingConfig\":null,\"ReverseGeocodingConfig\":{\"XAttributeName\":\"Longitude\",\"YAttributeName\":\"Latitude\"}}"
val para5: Parameter = Parameter {
name = "parameter_step_1_vej_config"
value = para5JSON
}
parameters.add(para1)
parameters.add(para2)
parameters.add(para3)
parameters.add(para4)
parameters.add(para5)
val pipelineExecutionRequest = StartPipelineExecutionRequest {
pipelineExecutionDescription = "Created using Kotlin SDK"
pipelineExecutionDisplayName = "$pipelineName-example-execution"
pipelineParameters = parameters
pipelineName = pipelineNameVal
}
SageMakerClient { region = "us-west-2" }.use { sageMakerClient ->
val response = sageMakerClient.startPipelineExecution(pipelineExecutionRequest)
return response.pipelineExecutionArn
}
}
```
+ API 세부 정보는 *AWS SDK for Kotlin API 참조*의 [StartPipelineExecution](https://sdk.amazonaws.com/kotlin/api/latest/index.html)를 참조하세요.
## 시나리오
### 지리공간 작업 및 파이프라인으로 시작하기
다음 코드 예제에서는 다음과 같은 작업을 수행하는 방법을 보여줍니다.
+ 파이프라인의 리소스를 설정하세요.
+ 지리 공간 작업을 실행하는 파이프라인을 설정합니다.
+ 파이프라인 실행을 시작합니다.
+ 실행 상태를 모니터링합니다.
+ 파이프라인의 출력을 볼 수 있습니다.
+ 리소스를 정리합니다.
자세한 내용은 [Community. AWS SDKs를 사용하여 SageMaker 파이프라인 생성 및 실행을 참조하세요](https://community.aws/posts/create-and-run-sagemaker-pipelines-using-aws-sdks).
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예제 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/usecases/workflow_sagemaker_pipes#code-examples)에서 전체 예제를 찾고 설정 및 실행하는 방법을 배워보세요.
```
val DASHES = String(CharArray(80)).replace("\u0000", "-")
private var eventSourceMapping = ""
suspend fun main(args: Array) {
val usage = """
Usage:
Where:
sageMakerRoleName - The name of the Amazon SageMaker role.
lambdaRoleName - The name of the AWS Lambda role.
functionName - The name of the AWS Lambda function (for example,SageMakerExampleFunction).
functionKey - The name of the Amazon S3 key name that represents the Lambda function (for example, SageMakerLambda.zip).
queueName - The name of the Amazon Simple Queue Service (Amazon SQS) queue.
bucketName - The name of the Amazon Simple Storage Service (Amazon S3) bucket.
bucketFunction - The name of the Amazon S3 bucket that contains the Lambda ZIP file.
lnglatData - The file location of the latlongtest.csv file required for this use case.
spatialPipelinePath - The file location of the GeoSpatialPipeline.json file required for this use case.
pipelineName - The name of the pipeline to create (for example, sagemaker-sdk-example-pipeline).
"""
if (args.size != 10) {
println(usage)
exitProcess(1)
}
val sageMakerRoleName = args[0]
val lambdaRoleName = args[1]
val functionKey = args[2]
val functionName = args[3]
val queueName = args[4]
val bucketName = args[5]
val bucketFunction = args[6]
val lnglatData = args[7]
val spatialPipelinePath = args[8]
val pipelineName = args[9]
val handlerName = "org.example.SageMakerLambdaFunction::handleRequest"
println(DASHES)
println("Welcome to the Amazon SageMaker pipeline example scenario.")
println(
"""
This example workflow will guide you through setting up and running an
Amazon SageMaker pipeline. The pipeline uses an AWS Lambda function and an
Amazon SQS Queue. It runs a vector enrichment reverse geocode job to
reverse geocode addresses in an input file and store the results in an export file.
""".trimIndent(),
)
println(DASHES)
println(DASHES)
println("First, we will set up the roles, functions, and queue needed by the SageMaker pipeline.")
val lambdaRoleArn: String = checkLambdaRole(lambdaRoleName)
val sageMakerRoleArn: String = checkSageMakerRole(sageMakerRoleName)
val functionArn = checkFunction(functionName, bucketFunction, functionKey, handlerName, lambdaRoleArn)
val queueUrl = checkQueue(queueName, functionName)
println(DASHES)
println(DASHES)
println("Setting up bucket $bucketName")
if (!checkBucket(bucketName)) {
setupBucket(bucketName)
println("Put $lnglatData into $bucketName")
val objectKey = "samplefiles/latlongtest.csv"
putS3Object(bucketName, objectKey, lnglatData)
}
println(DASHES)
println(DASHES)
println("Now we can create and run our pipeline.")
setupPipeline(spatialPipelinePath, sageMakerRoleArn, functionArn, pipelineName)
val pipelineExecutionARN = executePipeline(bucketName, queueUrl, sageMakerRoleArn, pipelineName)
println("The pipeline execution ARN value is $pipelineExecutionARN")
waitForPipelineExecution(pipelineExecutionARN)
println("Wait 30 secs to get output results $bucketName")
TimeUnit.SECONDS.sleep(30)
getOutputResults(bucketName)
println(DASHES)
println(DASHES)
println(
"""
The pipeline has completed. To view the pipeline and runs in SageMaker Studio, follow these instructions:
https://docs.aws.amazon.com/sagemaker/latest/dg/pipelines-studio.html
""".trimIndent(),
)
println(DASHES)
println(DASHES)
println("Do you want to delete the AWS resources used in this Workflow? (y/n)")
val `in` = Scanner(System.`in`)
val delResources = `in`.nextLine()
if (delResources.compareTo("y") == 0) {
println("Lets clean up the AWS resources. Wait 30 seconds")
TimeUnit.SECONDS.sleep(30)
deleteEventSourceMapping(functionName)
deleteSQSQueue(queueName)
listBucketObjects(bucketName)
deleteBucket(bucketName)
delLambdaFunction(functionName)
deleteLambdaRole(lambdaRoleName)
deleteSagemakerRole(sageMakerRoleName)
deletePipeline(pipelineName)
} else {
println("The AWS Resources were not deleted!")
}
println(DASHES)
println(DASHES)
println("SageMaker pipeline scenario is complete.")
println(DASHES)
}
// Delete a SageMaker pipeline by name.
suspend fun deletePipeline(pipelineNameVal: String) {
val pipelineRequest = DeletePipelineRequest {
pipelineName = pipelineNameVal
}
SageMakerClient { region = "us-west-2" }.use { sageMakerClient ->
sageMakerClient.deletePipeline(pipelineRequest)
println("*** Successfully deleted $pipelineNameVal")
}
}
suspend fun deleteSagemakerRole(roleNameVal: String) {
val sageMakerRolePolicies = getSageMakerRolePolicies()
IamClient { region = "us-west-2" }.use { iam ->
for (policy in sageMakerRolePolicies) {
// First the policy needs to be detached.
val rolePolicyRequest = DetachRolePolicyRequest {
policyArn = policy
roleName = roleNameVal
}
iam.detachRolePolicy(rolePolicyRequest)
}
// Delete the role.
val roleRequest = DeleteRoleRequest {
roleName = roleNameVal
}
iam.deleteRole(roleRequest)
println("*** Successfully deleted $roleNameVal")
}
}
suspend fun deleteLambdaRole(roleNameVal: String) {
val lambdaRolePolicies = getLambdaRolePolicies()
IamClient { region = "us-west-2" }.use { iam ->
for (policy in lambdaRolePolicies) {
// First the policy needs to be detached.
val rolePolicyRequest = DetachRolePolicyRequest {
policyArn = policy
roleName = roleNameVal
}
iam.detachRolePolicy(rolePolicyRequest)
}
// Delete the role.
val roleRequest = DeleteRoleRequest {
roleName = roleNameVal
}
iam.deleteRole(roleRequest)
println("*** Successfully deleted $roleNameVal")
}
}
suspend fun delLambdaFunction(myFunctionName: String) {
val request = DeleteFunctionRequest {
functionName = myFunctionName
}
LambdaClient { region = "us-west-2" }.use { awsLambda ->
awsLambda.deleteFunction(request)
println("$myFunctionName was deleted")
}
}
suspend fun deleteBucket(bucketName: String?) {
val request = DeleteBucketRequest {
bucket = bucketName
}
S3Client { region = "us-east-1" }.use { s3 ->
s3.deleteBucket(request)
println("The $bucketName was successfully deleted!")
}
}
suspend fun deleteBucketObjects(bucketName: String, objectName: String?) {
val toDelete = ArrayList()
val obId = ObjectIdentifier {
key = objectName
}
toDelete.add(obId)
val delOb = Delete {
objects = toDelete
}
val dor = DeleteObjectsRequest {
bucket = bucketName
delete = delOb
}
S3Client { region = "us-east-1" }.use { s3Client ->
s3Client.deleteObjects(dor)
println("*** $bucketName objects were deleted.")
}
}
suspend fun listBucketObjects(bucketNameVal: String) {
val listObjects = ListObjectsRequest {
bucket = bucketNameVal
}
S3Client { region = "us-east-1" }.use { s3Client ->
val res = s3Client.listObjects(listObjects)
val objects = res.contents
if (objects != null) {
for (myValue in objects) {
println("The name of the key is ${myValue.key}")
deleteBucketObjects(bucketNameVal, myValue.key)
}
}
}
}
// Delete the specific Amazon SQS queue.
suspend fun deleteSQSQueue(queueNameVal: String?) {
val getQueueRequest = GetQueueUrlRequest {
queueName = queueNameVal
}
SqsClient { region = "us-west-2" }.use { sqsClient ->
val urlVal = sqsClient.getQueueUrl(getQueueRequest).queueUrl
val deleteQueueRequest = DeleteQueueRequest {
queueUrl = urlVal
}
sqsClient.deleteQueue(deleteQueueRequest)
}
}
// Delete the queue event mapping.
suspend fun deleteEventSourceMapping(functionNameVal: String) {
if (eventSourceMapping.compareTo("") == 0) {
LambdaClient { region = "us-west-2" }.use { lambdaClient ->
val request = ListEventSourceMappingsRequest {
functionName = functionNameVal
}
val response = lambdaClient.listEventSourceMappings(request)
val eventList = response.eventSourceMappings
if (eventList != null) {
for (event in eventList) {
eventSourceMapping = event.uuid.toString()
}
}
}
}
val eventSourceMappingRequest = DeleteEventSourceMappingRequest {
uuid = eventSourceMapping
}
LambdaClient { region = "us-west-2" }.use { lambdaClient ->
lambdaClient.deleteEventSourceMapping(eventSourceMappingRequest)
println("The event mapping is deleted!")
}
}
// Reads the objects in the S3 bucket and displays the values.
private suspend fun readObject(bucketName: String, keyVal: String?) {
println("Output file contents: \n")
val objectRequest = GetObjectRequest {
bucket = bucketName
key = keyVal
}
S3Client { region = "us-east-1" }.use { s3Client ->
s3Client.getObject(objectRequest) { resp ->
val byteArray = resp.body?.toByteArray()
val text = byteArray?.let { String(it, StandardCharsets.UTF_8) }
println("Text output: $text")
}
}
}
// Display the results from the output directory.
suspend fun getOutputResults(bucketName: String?) {
println("Getting output results $bucketName.")
val listObjectsRequest = ListObjectsRequest {
bucket = bucketName
prefix = "outputfiles/"
}
S3Client { region = "us-east-1" }.use { s3Client ->
val response = s3Client.listObjects(listObjectsRequest)
val s3Objects: List