Where:
filter - The name of the subscription filter (for example, MyFilter).
logGroup - The name of the log group. (for example, testgroup).
""";
if (args.length != 2) {
System.out.println(usage);
System.exit(1);
}
String filter = args[0];
String logGroup = args[1];
CloudWatchLogsClient logs = CloudWatchLogsClient.builder()
.build();
deleteSubFilter(logs, filter, logGroup);
logs.close();
}
public static void deleteSubFilter(CloudWatchLogsClient logs, String filter, String logGroup) {
try {
DeleteSubscriptionFilterRequest request = DeleteSubscriptionFilterRequest.builder()
.filterName(filter)
.logGroupName(logGroup)
.build();
logs.deleteSubscriptionFilter(request);
System.out.printf("Successfully deleted CloudWatch logs subscription filter %s", filter);
} catch (CloudWatchException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ API 세부 정보는 *AWS SDK for Java 2.x API 참조*의 [DeleteSubscriptionFilter](https://docs.aws.amazon.com/goto/SdkForJavaV2/logs-2014-03-28/DeleteSubscriptionFilter)를 참조하세요.
------
#### [ JavaScript ]
**SDK for JavaScript(v3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
import { DeleteSubscriptionFilterCommand } from "@aws-sdk/client-cloudwatch-logs";
import { client } from "../libs/client.js";
const run = async () => {
const command = new DeleteSubscriptionFilterCommand({
// The name of the filter.
filterName: process.env.CLOUDWATCH_LOGS_FILTER_NAME,
// The name of the log group.
logGroupName: process.env.CLOUDWATCH_LOGS_LOG_GROUP,
});
try {
return await client.send(command);
} catch (err) {
console.error(err);
}
};
export default run();
```
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [DeleteSubscriptionFilter](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/DeleteSubscriptionFilterCommand)를 참조하세요.
**SDK for JavaScript(v2)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascript/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");
// Set the region
AWS.config.update({ region: "REGION" });
// Create the CloudWatchLogs service object
var cwl = new AWS.CloudWatchLogs({ apiVersion: "2014-03-28" });
var params = {
filterName: "FILTER",
logGroupName: "LOG_GROUP",
};
cwl.deleteSubscriptionFilter(params, function (err, data) {
if (err) {
console.log("Error", err);
} else {
console.log("Success", data);
}
});
```
+ 자세한 정보는 [AWS SDK for JavaScript 개발자 안내서](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/cloudwatch-examples-subscriptions.html#cloudwatch-examples-subscriptions-deleting)를 참조하세요.
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [DeleteSubscriptionFilter](https://docs.aws.amazon.com/goto/AWSJavaScriptSDK/logs-2014-03-28/DeleteSubscriptionFilter)를 참조하세요.
------
#### [ Kotlin ]
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/services/cloudwatch#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
suspend fun deleteSubFilter(
filter: String?,
logGroup: String?,
) {
val request =
DeleteSubscriptionFilterRequest {
filterName = filter
logGroupName = logGroup
}
CloudWatchLogsClient.fromEnvironment { region = "us-west-2" }.use { logs ->
logs.deleteSubscriptionFilter(request)
println("Successfully deleted CloudWatch logs subscription filter named $filter")
}
}
```
+ API 세부 정보는 *AWS SDK for Kotlin API 참조*의 [DeleteSubscriptionFilter](https://sdk.amazonaws.com/kotlin/api/latest/index.html)를 참조하세요.
------
# AWS SDK와 `DescribeExportTasks` 함께 사용
다음 코드 예시는 `DescribeExportTasks`의 사용 방법을 보여 줍니다.
------
#### [ .NET ]
**SDK for .NET**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/CloudWatchLogs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
using System;
using System.Threading.Tasks;
using Amazon.CloudWatchLogs;
using Amazon.CloudWatchLogs.Model;
///
/// Shows how to retrieve a list of information about Amazon CloudWatch
/// Logs export tasks.
///
public class DescribeExportTasks
{
public static async Task Main()
{
// This client object will be associated with the same AWS Region
// as the default user on this system. If you need to use a
// different AWS Region, pass it as a parameter to the client
// constructor.
var client = new AmazonCloudWatchLogsClient();
var request = new DescribeExportTasksRequest
{
Limit = 5,
};
var response = new DescribeExportTasksResponse();
do
{
response = await client.DescribeExportTasksAsync(request);
response.ExportTasks.ForEach(t =>
{
Console.WriteLine($"{t.TaskName} with ID: {t.TaskId} has status: {t.Status}");
});
}
while (response.NextToken is not null);
}
}
```
+ API 세부 정보는 *AWS SDK for .NET API 참조*의 [DescribeExportTasks](https://docs.aws.amazon.com/goto/DotNetSDKV3/logs-2014-03-28/DescribeExportTasks) 참조하세요.
------
# AWS SDK 또는 CLI와 `DescribeLogGroups` 함께 사용
다음 코드 예시는 `DescribeLogGroups`의 사용 방법을 보여 줍니다.
------
#### [ .NET ]
**SDK for .NET**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv3/CloudWatchLogs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
using System;
using System.Threading.Tasks;
using Amazon.CloudWatchLogs;
using Amazon.CloudWatchLogs.Model;
///
/// Retrieves information about existing Amazon CloudWatch Logs log groups
/// and displays the information on the console.
///
public class DescribeLogGroups
{
public static async Task Main()
{
// Creates a CloudWatch Logs client using the default
// user. If you need to work with resources in another
// AWS Region than the one defined for the default user,
// pass the AWS Region as a parameter to the client constructor.
var client = new AmazonCloudWatchLogsClient();
bool done = false;
string newToken = null;
var request = new DescribeLogGroupsRequest
{
Limit = 5,
};
DescribeLogGroupsResponse response;
do
{
if (newToken is not null)
{
request.NextToken = newToken;
}
response = await client.DescribeLogGroupsAsync(request);
response.LogGroups.ForEach(lg =>
{
Console.WriteLine($"{lg.LogGroupName} is associated with the key: {lg.KmsKeyId}.");
Console.WriteLine($"Created on: {lg.CreationTime.Date.Date}");
Console.WriteLine($"Date for this group will be stored for: {lg.RetentionInDays} days.\n");
});
if (response.NextToken is null)
{
done = true;
}
else
{
newToken = response.NextToken;
}
}
while (!done);
}
}
```
+ API 세부 정보는 *AWS SDK for .NET API 참조*의 [DescribeLogGroups](https://docs.aws.amazon.com/goto/DotNetSDKV3/logs-2014-03-28/DescribeLogGroups)를 참조하세요.
------
#### [ CLI ]
**AWS CLI**
다음 명령은 이름이 `my-logs`인 로그 그룹을 설명합니다.
```
aws logs describe-log-groups --log-group-name-prefix my-logs
```
출력:
```
{
"logGroups": [
{
"storedBytes": 0,
"metricFilterCount": 0,
"creationTime": 1433189500783,
"logGroupName": "my-logs",
"retentionInDays": 5,
"arn": "arn:aws:logs:us-west-2:0123456789012:log-group:my-logs:*"
}
]
}
```
+ API 세부 정보는 **AWS CLI 명령 참조의 [DescribeLogGroups](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/logs/describe-log-groups.html)를 참조하세요.
------
#### [ JavaScript ]
**SDK for JavaScript (v3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
import {
paginateDescribeLogGroups,
CloudWatchLogsClient,
} from "@aws-sdk/client-cloudwatch-logs";
const client = new CloudWatchLogsClient({});
export const main = async () => {
const paginatedLogGroups = paginateDescribeLogGroups({ client }, {});
const logGroups = [];
for await (const page of paginatedLogGroups) {
if (page.logGroups?.every((lg) => !!lg)) {
logGroups.push(...page.logGroups);
}
}
console.log(logGroups);
return logGroups;
};
```
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [DescribeLogGroups](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/DescribeLogGroupsCommand) 참조하세요.
------
# AWS SDK 또는 CLI와 `DescribeLogStreams` 함께 사용
다음 코드 예시는 `DescribeLogStreams`의 사용 방법을 보여 줍니다.
------
#### [ CLI ]
**AWS CLI**
다음 명령은 로그 그룹 `my-logs`의 `2015` 접두사로 시작하는 모든 로그 스트림을 보여 줍니다.
```
aws logs describe-log-streams --log-group-name my-logs --log-stream-name-prefix 2015
```
출력:
```
{
"logStreams": [
{
"creationTime": 1433189871774,
"arn": "arn:aws:logs:us-west-2:0123456789012:log-group:my-logs:log-stream:20150531",
"logStreamName": "20150531",
"storedBytes": 0
},
{
"creationTime": 1433189873898,
"arn": "arn:aws:logs:us-west-2:0123456789012:log-group:my-logs:log-stream:20150601",
"logStreamName": "20150601",
"storedBytes": 0
}
]
}
```
+ API 세부 정보는 *AWS CLI 명령 참조*의 [DescribeLogStreams](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/logs/describe-log-streams.html)를 참조하세요.
------
#### [ Java ]
**SDK for Java 2.x**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
지정된 접두사와 일치하는 지정된 로그 그룹 내의 로그 스트림을 검색합니다.
```
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class CloudWatchLogsSearch {
public static void main(String[] args) {
final String usage = """
Usage:
Where:
logGroupName - The name of the log group (for example, WeathertopJavaContainerLogs).
logStreamName - The name of the log stream (for example, weathertop-java-stream).
pattern - the pattern to use (for example, INFO)
""";
if (args.length != 3) {
System.out.print(usage);
System.exit(1);
}
String logGroupName = args[0] ;
String logStreamName = args[1] ;
String pattern = args[2] ;
CloudWatchLogsClient cwlClient = CloudWatchLogsClient.builder()
.region(Region.US_EAST_1)
.build();
searchLogStreamsAndFilterEvents(cwlClient, logGroupName, logStreamName, pattern);
}
/**
* Searches for log streams with a specific prefix within a log group and filters log events based on a specified pattern.
*
* @param cwlClient the CloudWatchLogsClient used to interact with AWS CloudWatch Logs
* @param logGroupName the name of the log group to search within
* @param logStreamPrefix the prefix of the log streams to search for
* @param pattern the pattern to filter log events by
*/
public static void searchLogStreamsAndFilterEvents(CloudWatchLogsClient cwlClient, String logGroupName, String logStreamPrefix, String pattern) {
DescribeLogStreamsRequest describeLogStreamsRequest = DescribeLogStreamsRequest.builder()
.logGroupName(logGroupName)
.logStreamNamePrefix(logStreamPrefix)
.build();
DescribeLogStreamsResponse describeLogStreamsResponse = cwlClient.describeLogStreams(describeLogStreamsRequest);
List logStreams = describeLogStreamsResponse.logStreams();
for (LogStream logStream : logStreams) {
String logStreamName = logStream.logStreamName();
System.out.println("Searching in log stream: " + logStreamName);
FilterLogEventsRequest filterLogEventsRequest = FilterLogEventsRequest.builder()
.logGroupName(logGroupName)
.logStreamNames(logStreamName)
.filterPattern(pattern)
.build();
FilterLogEventsResponse filterLogEventsResponse = cwlClient.filterLogEvents(filterLogEventsRequest);
for (FilteredLogEvent event : filterLogEventsResponse.events()) {
System.out.println(event.message());
}
System.out.println("--------------------------------------------------"); // Separator for better readability
}
}
}
```
지정된 로그 그룹의 최신 로그 스트림에 대한 메타데이터를 출력합니다.
```
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class CloudWatchLogQuery {
public static void main(final String[] args) {
final String usage = """
Usage:
Where:
logGroupName - The name of the log group (for example, /aws/lambda/ChatAIHandler).
""";
if (args.length != 1) {
System.out.print(usage);
System.exit(1);
}
String logGroupName = "/aws/lambda/ChatAIHandler" ; //args[0];
CloudWatchLogsClient logsClient = CloudWatchLogsClient.builder()
.region(Region.US_EAST_1)
.build();
describeMostRecentLogStream(logsClient, logGroupName);
}
/**
* Describes and prints metadata about the most recent log stream in the specified log group.
*
* @param logsClient the CloudWatchLogsClient used to interact with AWS CloudWatch Logs
* @param logGroupName the name of the log group
*/
public static void describeMostRecentLogStream(CloudWatchLogsClient logsClient, String logGroupName) {
DescribeLogStreamsRequest streamsRequest = DescribeLogStreamsRequest.builder()
.logGroupName(logGroupName)
.orderBy(OrderBy.LAST_EVENT_TIME)
.descending(true)
.limit(1)
.build();
try {
DescribeLogStreamsResponse streamsResponse = logsClient.describeLogStreams(streamsRequest);
List logStreams = streamsResponse.logStreams();
if (logStreams.isEmpty()) {
System.out.println("No log streams found for log group: " + logGroupName);
return;
}
LogStream stream = logStreams.get(0);
System.out.println("Most Recent Log Stream:");
System.out.println(" Name: " + stream.logStreamName());
System.out.println(" ARN: " + stream.arn());
System.out.println(" Creation Time: " + stream.creationTime());
System.out.println(" First Event Time: " + stream.firstEventTimestamp());
System.out.println(" Last Event Time: " + stream.lastEventTimestamp());
System.out.println(" Stored Bytes: " + stream.storedBytes());
System.out.println(" Upload Sequence Token: " + stream.uploadSequenceToken());
} catch (CloudWatchLogsException e) {
System.err.println("Failed to describe log stream: " + e.awsErrorDetails().errorMessage());
}
}
}
```
+ API 세부 정보는 *AWS SDK for Java 2.x API 참조*의 [DescribeLogStreams](https://docs.aws.amazon.com/goto/SdkForJavaV2/logs-2014-03-28/DescribeLogStreams)을 참조하세요.
------
# AWS SDK와 `DescribeSubscriptionFilters` 함께 사용
다음 코드 예시는 `DescribeSubscriptionFilters`의 사용 방법을 보여 줍니다.
------
#### [ C\$1\$1 ]
**SDK for C\$1\$1**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/cpp/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
필수 파일을 포함합니다.
```
#include
#include
#include
#include
#include
#include
#include
```
구독 필터를 나열합니다.
```
Aws::CloudWatchLogs::CloudWatchLogsClient cwl;
Aws::CloudWatchLogs::Model::DescribeSubscriptionFiltersRequest request;
request.SetLogGroupName(log_group);
request.SetLimit(1);
bool done = false;
bool header = false;
while (!done) {
auto outcome = cwl.DescribeSubscriptionFilters(
request);
if (!outcome.IsSuccess()) {
std::cout << "Failed to describe CloudWatch subscription filters "
<< "for log group " << log_group << ": " <<
outcome.GetError().GetMessage() << std::endl;
break;
}
if (!header) {
std::cout << std::left << std::setw(32) << "Name" <<
std::setw(64) << "FilterPattern" << std::setw(64) <<
"DestinationArn" << std::endl;
header = true;
}
const auto &filters = outcome.GetResult().GetSubscriptionFilters();
for (const auto &filter : filters) {
std::cout << std::left << std::setw(32) <<
filter.GetFilterName() << std::setw(64) <<
filter.GetFilterPattern() << std::setw(64) <<
filter.GetDestinationArn() << std::endl;
}
const auto &next_token = outcome.GetResult().GetNextToken();
request.SetNextToken(next_token);
done = next_token.empty();
}
```
+ API 세부 정보는 *AWS SDK for C\$1\$1 API 참조*의 [DescribeSubscriptionFilters](https://docs.aws.amazon.com/goto/SdkForCpp/logs-2014-03-28/DescribeSubscriptionFilters)를 참조하세요.
------
#### [ Java ]
**SDK for Java 2.x**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.services.cloudwatch.model.CloudWatchException;
import software.amazon.awssdk.services.cloudwatchlogs.CloudWatchLogsClient;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeSubscriptionFiltersRequest;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeSubscriptionFiltersResponse;
import software.amazon.awssdk.services.cloudwatchlogs.model.SubscriptionFilter;
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class DescribeSubscriptionFilters {
public static void main(String[] args) {
final String usage = """
Usage:
Where:
logGroup - A log group name (for example, myloggroup).
""";
if (args.length != 1) {
System.out.println(usage);
System.exit(1);
}
String logGroup = args[0];
CloudWatchLogsClient logs = CloudWatchLogsClient.builder()
.credentialsProvider(ProfileCredentialsProvider.create())
.build();
describeFilters(logs, logGroup);
logs.close();
}
public static void describeFilters(CloudWatchLogsClient logs, String logGroup) {
try {
boolean done = false;
String newToken = null;
while (!done) {
DescribeSubscriptionFiltersResponse response;
if (newToken == null) {
DescribeSubscriptionFiltersRequest request = DescribeSubscriptionFiltersRequest.builder()
.logGroupName(logGroup)
.limit(1).build();
response = logs.describeSubscriptionFilters(request);
} else {
DescribeSubscriptionFiltersRequest request = DescribeSubscriptionFiltersRequest.builder()
.nextToken(newToken)
.logGroupName(logGroup)
.limit(1).build();
response = logs.describeSubscriptionFilters(request);
}
for (SubscriptionFilter filter : response.subscriptionFilters()) {
System.out.printf("Retrieved filter with name %s, " + "pattern %s " + "and destination arn %s",
filter.filterName(),
filter.filterPattern(),
filter.destinationArn());
}
if (response.nextToken() == null) {
done = true;
} else {
newToken = response.nextToken();
}
}
} catch (CloudWatchException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
System.out.printf("Done");
}
}
```
+ API 세부 정보는 *AWS SDK for Java 2.x API 참조*의 [DescribeSubscriptionFilters](https://docs.aws.amazon.com/goto/SdkForJavaV2/logs-2014-03-28/DescribeSubscriptionFilters)를 참조하세요.
------
#### [ JavaScript ]
**SDK for JavaScript(v3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
import { DescribeSubscriptionFiltersCommand } from "@aws-sdk/client-cloudwatch-logs";
import { client } from "../libs/client.js";
const run = async () => {
// This will return a list of all subscription filters in your account
// matching the log group name.
const command = new DescribeSubscriptionFiltersCommand({
logGroupName: process.env.CLOUDWATCH_LOGS_LOG_GROUP,
limit: 1,
});
try {
return await client.send(command);
} catch (err) {
console.error(err);
}
};
export default run();
```
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [DescribeSubscriptionFilters](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/DescribeSubscriptionFiltersCommand)를 참조하세요.
**SDK for JavaScript(v2)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascript/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");
// Set the region
AWS.config.update({ region: "REGION" });
// Create the CloudWatchLogs service object
var cwl = new AWS.CloudWatchLogs({ apiVersion: "2014-03-28" });
var params = {
logGroupName: "GROUP_NAME",
limit: 5,
};
cwl.describeSubscriptionFilters(params, function (err, data) {
if (err) {
console.log("Error", err);
} else {
console.log("Success", data.subscriptionFilters);
}
});
```
+ 자세한 정보는 [AWS SDK for JavaScript 개발자 안내서](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/cloudwatch-examples-subscriptions.html#cloudwatch-examples-subscriptions-describing)를 참조하세요.
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [DescribeSubscriptionFilters](https://docs.aws.amazon.com/goto/AWSJavaScriptSDK/logs-2014-03-28/DescribeSubscriptionFilters)를 참조하세요.
------
#### [ Kotlin ]
**SDK for Kotlin**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/kotlin/services/cloudwatch#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
suspend fun describeFilters(logGroup: String) {
val request =
DescribeSubscriptionFiltersRequest {
logGroupName = logGroup
limit = 1
}
CloudWatchLogsClient.fromEnvironment { region = "us-west-2" }.use { cwlClient ->
val response = cwlClient.describeSubscriptionFilters(request)
response.subscriptionFilters?.forEach { filter ->
println("Retrieved filter with name ${filter.filterName} pattern ${filter.filterPattern} and destination ${filter.destinationArn}")
}
}
}
```
+ API 세부 정보는 *AWS SDK for Kotlin API 참조*의 [DescribeSubscriptionFilters](https://sdk.amazonaws.com/kotlin/api/latest/index.html)를 참조하세요.
------
# AWS SDK 또는 CLI와 `GetLogEvents` 함께 사용
다음 코드 예시는 `GetLogEvents`의 사용 방법을 보여 줍니다.
------
#### [ CLI ]
**AWS CLI**
다음 명령은 `my-logs` 로그 그룹에서 이름이 `20150601`인 로그 스트림의 로그 이벤트를 검색합니다.
```
aws logs get-log-events --log-group-name my-logs --log-stream-name 20150601
```
출력:
```
{
"nextForwardToken": "f/31961209122447488583055879464742346735121166569214640130",
"events": [
{
"ingestionTime": 1433190494190,
"timestamp": 1433190184356,
"message": "Example Event 1"
},
{
"ingestionTime": 1433190516679,
"timestamp": 1433190184356,
"message": "Example Event 1"
},
{
"ingestionTime": 1433190494190,
"timestamp": 1433190184358,
"message": "Example Event 2"
}
],
"nextBackwardToken": "b/31961209122358285602261756944988674324553373268216709120"
}
```
+ API 세부 정보는 *AWS CLI 명령 참조*의 [GetLogEvents](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/logs/get-log-events.html)를 참조하세요.
------
#### [ Java ]
**SDK for Java 2.x**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/cloudwatch#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.cloudwatch.model.CloudWatchException;
import software.amazon.awssdk.services.cloudwatchlogs.CloudWatchLogsClient;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeLogStreamsRequest;
import software.amazon.awssdk.services.cloudwatchlogs.model.DescribeLogStreamsResponse;
import software.amazon.awssdk.services.cloudwatchlogs.model.GetLogEventsRequest;
import software.amazon.awssdk.services.cloudwatchlogs.model.GetLogEventsResponse;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
/**
* Before running this Java V2 code example, set up your development
* environment, including your credentials.
*
* For more information, see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class GetLogEvents {
public static void main(String[] args) {
final String usage = """
Usage:
Where:
logGroupName - The name of the log group (for example, myloggroup).
logStreamName - The name of the log stream (for example, mystream).
""";
// if (args.length != 2) {
// System.out.print(usage);
// System.exit(1);
// }
String logGroupName = "WeathertopJavaContainerLogs" ; //args[0];
String logStreamName = "weathertop-java-stream" ; //args[1];
Region region = Region.US_EAST_1 ;
CloudWatchLogsClient cloudWatchLogsClient = CloudWatchLogsClient.builder()
.region(region)
.build();
getCWLogEvents(cloudWatchLogsClient, logGroupName, logStreamName);
cloudWatchLogsClient.close();
}
public static void getCWLogEvents(CloudWatchLogsClient cloudWatchLogsClient,
String logGroupName,
String logStreamPrefix) {
try {
// First, find the exact log stream name
DescribeLogStreamsRequest describeRequest = DescribeLogStreamsRequest.builder()
.logGroupName(logGroupName)
.logStreamNamePrefix(logStreamPrefix)
.limit(1) // get the first matching stream
.build();
DescribeLogStreamsResponse describeResponse = cloudWatchLogsClient.describeLogStreams(describeRequest);
if (describeResponse.logStreams().isEmpty()) {
System.out.println("No matching log streams found for prefix: " + logStreamPrefix);
return;
}
String exactLogStreamName = describeResponse.logStreams().get(0).logStreamName();
System.out.println("Using exact log stream: " + exactLogStreamName);
long startTime = Instant.now().minus(7, ChronoUnit.DAYS).toEpochMilli();
long endTime = Instant.now().toEpochMilli();
GetLogEventsRequest getLogEventsRequest = GetLogEventsRequest.builder()
.logGroupName(logGroupName)
.logStreamName(exactLogStreamName) // <-- exact name, not prefix
.startTime(startTime)
.endTime(endTime)
.startFromHead(true)
.build();
GetLogEventsResponse response = cloudWatchLogsClient.getLogEvents(getLogEventsRequest);
if (response.events().isEmpty()) {
System.out.println("No log events found in the past 7 days.");
} else {
response.events().forEach(e -> System.out.println(e.message()));
}
} catch (CloudWatchException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ API 세부 정보는 *AWS SDK for Java 2.x API 참조*의 [GetLogEvents](https://docs.aws.amazon.com/goto/SdkForJavaV2/logs-2014-03-28/GetLogEvents)를 참조하세요.
------
# AWS SDK와 `GetQueryResults` 함께 사용
다음 코드 예시는 `GetQueryResults`의 사용 방법을 보여 줍니다.
작업 예제는 대규모 프로그램에서 발췌한 코드이며 컨텍스트에 맞춰 실행해야 합니다. 다음 코드 예제에서는 컨텍스트 내에서 이 작업을 확인할 수 있습니다.
+ [대용량 쿼리 실행](cloudwatch-logs_example_cloudwatch-logs_Scenario_BigQuery_section.md)
------
#### [ .NET ]
**SDK for .NET (v4)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv4/CloudWatchLogs/LargeQuery#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
///
/// Gets the results of a CloudWatch Logs Insights query.
///
/// The ID of the query.
/// The query results response.
public async Task GetQueryResultsAsync(string queryId)
{
try
{
var request = new GetQueryResultsRequest
{
QueryId = queryId
};
var response = await _amazonCloudWatchLogs.GetQueryResultsAsync(request);
return response;
}
catch (ResourceNotFoundException ex)
{
_logger.LogError($"Query not found: {ex.Message}");
return null;
}
catch (Exception ex)
{
_logger.LogError($"An error occurred while getting query results: {ex.Message}");
return null;
}
}
```
+ API 세부 정보는 *AWS SDK for .NET API 참조*의 [GetQueryResults](https://docs.aws.amazon.com/goto/DotNetSDKV4/logs-2014-03-28/GetQueryResults)를 참조하세요.
------
#### [ JavaScript ]
**SDK for JavaScript (v3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
/**
* Simple wrapper for the GetQueryResultsCommand.
* @param {string} queryId
*/
_getQueryResults(queryId) {
return this.client.send(new GetQueryResultsCommand({ queryId }));
}
```
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [GetQueryResults](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/GetQueryResultsCommand)를 참조하세요.
------
#### [ Python ]
**SDK for Python(Boto3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
def _wait_for_query_results(self, client, query_id):
"""
Waits for the query to complete and retrieves the results.
:param query_id: The ID of the initiated query.
:type query_id: str
:return: A list containing the results of the query.
:rtype: list
"""
while True:
time.sleep(1)
results = client.get_query_results(queryId=query_id)
if results["status"] in [
"Complete",
"Failed",
"Cancelled",
"Timeout",
"Unknown",
]:
return results.get("results", [])
```
+ API 세부 정보는 [AWS SDK for Python(Boto3) API 참조](https://docs.aws.amazon.com/goto/boto3/logs-2014-03-28/GetQueryResults)의 *GetQueryResults*를 참조하세요.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP API**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cwl#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
TRY.
oo_result = lo_cwl->getqueryresults(
iv_queryid = iv_query_id ).
" Display query status and result count
DATA(lv_status) = oo_result->get_status( ).
DATA(lt_results) = oo_result->get_results( ).
DATA(lv_result_count) = lines( lt_results ).
MESSAGE |Query status: { lv_status }. Retrieved { lv_result_count } log event(s).| TYPE 'I'.
CATCH /aws1/cx_cwlinvalidparameterex.
MESSAGE 'Invalid parameter.' TYPE 'E'.
CATCH /aws1/cx_cwlresourcenotfoundex.
MESSAGE 'Resource not found.' TYPE 'E'.
CATCH /aws1/cx_cwlserviceunavailex.
MESSAGE 'Service unavailable.' TYPE 'E'.
ENDTRY.
```
+ API 세부 정보는 SDK for SAP ABAP API 참조의 [GetQueryResults](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html)를 참조하세요. *AWS *
------
# AWS SDK와 `PutSubscriptionFilter` 함께 사용
다음 코드 예시는 `PutSubscriptionFilter`의 사용 방법을 보여 줍니다.
------
#### [ C\$1\$1 ]
**SDK for C\$1\$1**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/cpp/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
필수 파일을 포함합니다.
```
#include
#include
#include
#include
#include
```
구독 필터를 생성합니다.
```
Aws::CloudWatchLogs::CloudWatchLogsClient cwl;
Aws::CloudWatchLogs::Model::PutSubscriptionFilterRequest request;
request.SetFilterName(filter_name);
request.SetFilterPattern(filter_pattern);
request.SetLogGroupName(log_group);
request.SetDestinationArn(dest_arn);
auto outcome = cwl.PutSubscriptionFilter(request);
if (!outcome.IsSuccess())
{
std::cout << "Failed to create CloudWatch logs subscription filter "
<< filter_name << ": " << outcome.GetError().GetMessage() <<
std::endl;
}
else
{
std::cout << "Successfully created CloudWatch logs subscription " <<
"filter " << filter_name << std::endl;
}
```
+ API 세부 정보는 *AWS SDK for C\$1\$1 API 참조*의 [PutSubscriptionFilter](https://docs.aws.amazon.com/goto/SdkForCpp/logs-2014-03-28/PutSubscriptionFilter)를 참조하세요.
------
#### [ Java ]
**SDK for Java 2.x**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.cloudwatchlogs.CloudWatchLogsClient;
import software.amazon.awssdk.services.cloudwatchlogs.model.CloudWatchLogsException;
import software.amazon.awssdk.services.cloudwatchlogs.model.PutSubscriptionFilterRequest;
/**
* Before running this code example, you need to grant permission to CloudWatch
* Logs the right to execute your Lambda function.
* To perform this task, you can use this CLI command:
*
* aws lambda add-permission --function-name "lamda1" --statement-id "lamda1"
* --principal "logs.us-west-2.amazonaws.com" --action "lambda:InvokeFunction"
* --source-arn "arn:aws:logs:us-west-2:111111111111:log-group:testgroup:*"
* --source-account "111111111111"
*
* Make sure you replace the function name with your function name and replace
* '111111111111' with your account details.
* For more information, see "Subscription Filters with AWS Lambda" in the
* Amazon CloudWatch Logs Guide.
*
*
* Also, before running this Java V2 code example,set up your development
* environment,including your credentials.
*
* For more information,see the following documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*
*/
public class PutSubscriptionFilter {
public static void main(String[] args) {
final String usage = """
Usage:
\s
Where:
filter - A filter name (for example, myfilter).
pattern - A filter pattern (for example, ERROR).
logGroup - A log group name (testgroup).
functionArn - An AWS Lambda function ARN (for example, arn:aws:lambda:us-west-2:111111111111:function:lambda1) .
""";
if (args.length != 4) {
System.out.println(usage);
System.exit(1);
}
String filter = args[0];
String pattern = args[1];
String logGroup = args[2];
String functionArn = args[3];
Region region = Region.US_WEST_2;
CloudWatchLogsClient cwl = CloudWatchLogsClient.builder()
.region(region)
.build();
putSubFilters(cwl, filter, pattern, logGroup, functionArn);
cwl.close();
}
public static void putSubFilters(CloudWatchLogsClient cwl,
String filter,
String pattern,
String logGroup,
String functionArn) {
try {
PutSubscriptionFilterRequest request = PutSubscriptionFilterRequest.builder()
.filterName(filter)
.filterPattern(pattern)
.logGroupName(logGroup)
.destinationArn(functionArn)
.build();
cwl.putSubscriptionFilter(request);
System.out.printf(
"Successfully created CloudWatch logs subscription filter %s",
filter);
} catch (CloudWatchLogsException e) {
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
}
}
}
```
+ API 세부 정보는 *AWS SDK for Java 2.x API 참조*의 [PutSubscriptionFilter](https://docs.aws.amazon.com/goto/SdkForJavaV2/logs-2014-03-28/PutSubscriptionFilter)를 참조하세요.
------
#### [ JavaScript ]
**SDK for JavaScript(v3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
import { PutSubscriptionFilterCommand } from "@aws-sdk/client-cloudwatch-logs";
import { client } from "../libs/client.js";
const run = async () => {
const command = new PutSubscriptionFilterCommand({
// An ARN of a same-account Kinesis stream, Kinesis Firehose
// delivery stream, or Lambda function.
// https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html
destinationArn: process.env.CLOUDWATCH_LOGS_DESTINATION_ARN,
// A name for the filter.
filterName: process.env.CLOUDWATCH_LOGS_FILTER_NAME,
// A filter pattern for subscribing to a filtered stream of log events.
// https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html
filterPattern: process.env.CLOUDWATCH_LOGS_FILTER_PATTERN,
// The name of the log group. Messages in this group matching the filter pattern
// will be sent to the destination ARN.
logGroupName: process.env.CLOUDWATCH_LOGS_LOG_GROUP,
});
try {
return await client.send(command);
} catch (err) {
console.error(err);
}
};
export default run();
```
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [PutSubscriptionFilter](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/PutSubscriptionFilterCommand)를 참조하세요.
**SDK for JavaScript(v2)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascript/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
// Load the AWS SDK for Node.js
var AWS = require("aws-sdk");
// Set the region
AWS.config.update({ region: "REGION" });
// Create the CloudWatchLogs service object
var cwl = new AWS.CloudWatchLogs({ apiVersion: "2014-03-28" });
var params = {
destinationArn: "LAMBDA_FUNCTION_ARN",
filterName: "FILTER_NAME",
filterPattern: "ERROR",
logGroupName: "LOG_GROUP",
};
cwl.putSubscriptionFilter(params, function (err, data) {
if (err) {
console.log("Error", err);
} else {
console.log("Success", data);
}
});
```
+ 자세한 정보는 [AWS SDK for JavaScript 개발자 안내서](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/cloudwatch-examples-subscriptions.html#cloudwatch-examples-subscriptions-creating)를 참조하세요.
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [PutSubscriptionFilter](https://docs.aws.amazon.com/goto/AWSJavaScriptSDK/logs-2014-03-28/PutSubscriptionFilter) 참조하세요.
------
# AWS SDK와 `StartLiveTail` 함께 사용
다음 코드 예시는 `StartLiveTail`의 사용 방법을 보여 줍니다.
------
#### [ .NET ]
**SDK for .NET**
필수 파일을 포함합니다.
```
using Amazon;
using Amazon.CloudWatchLogs;
using Amazon.CloudWatchLogs.Model;
```
Live Tail 세션을 시작합니다.
```
var client = new AmazonCloudWatchLogsClient();
var request = new StartLiveTailRequest
{
LogGroupIdentifiers = logGroupIdentifiers,
LogStreamNames = logStreamNames,
LogEventFilterPattern = filterPattern,
};
var response = await client.StartLiveTailAsync(request);
// Catch if request fails
if (response.HttpStatusCode != System.Net.HttpStatusCode.OK)
{
Console.WriteLine("Failed to start live tail session");
return;
}
```
Live Tail 세션의 이벤트는 두 가지 방법으로 처리할 수 있습니다.
```
/* Method 1
* 1). Asynchronously loop through the event stream
* 2). Set a timer to dispose the stream and stop the Live Tail session at the end.
*/
var eventStream = response.ResponseStream;
var task = Task.Run(() =>
{
foreach (var item in eventStream)
{
if (item is LiveTailSessionUpdate liveTailSessionUpdate)
{
foreach (var sessionResult in liveTailSessionUpdate.SessionResults)
{
Console.WriteLine("Message : {0}", sessionResult.Message);
}
}
if (item is LiveTailSessionStart)
{
Console.WriteLine("Live Tail session started");
}
// On-stream exceptions are processed here
if (item is CloudWatchLogsEventStreamException)
{
Console.WriteLine($"ERROR: {item}");
}
}
});
// Close the stream to stop the session after a timeout
if (!task.Wait(TimeSpan.FromSeconds(10))){
eventStream.Dispose();
Console.WriteLine("End of line");
}
```
```
/* Method 2
* 1). Add event handlers to each event variable
* 2). Start processing the stream and wait for a timeout using AutoResetEvent
*/
AutoResetEvent endEvent = new AutoResetEvent(false);
var eventStream = response.ResponseStream;
using (eventStream) // automatically disposes the stream to stop the session after execution finishes
{
eventStream.SessionStartReceived += (sender, e) =>
{
Console.WriteLine("LiveTail session started");
};
eventStream.SessionUpdateReceived += (sender, e) =>
{
foreach (LiveTailSessionLogEvent logEvent in e.EventStreamEvent.SessionResults){
Console.WriteLine("Message: {0}", logEvent.Message);
}
};
// On-stream exceptions are captured here
eventStream.ExceptionReceived += (sender, e) =>
{
Console.WriteLine($"ERROR: {e.EventStreamException.Message}");
};
eventStream.StartProcessing();
// Stream events for this amount of time.
endEvent.WaitOne(TimeSpan.FromSeconds(10));
Console.WriteLine("End of line");
}
```
+ API 세부 정보는 **AWS SDK for .NET API 참조의 [StartLiveTail](https://docs.aws.amazon.com/goto/DotNetSDKV3/logs-2014-03-28/StartLiveTail)을 참조하세요.
------
#### [ Go ]
**SDK for Go V2**
필수 파일을 포함합니다.
```
import (
"context"
"log"
"time"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs"
"github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs/types"
)
```
Live Tail 세션의 이벤트를 처리합니다.
```
func handleEventStreamAsync(stream *cloudwatchlogs.StartLiveTailEventStream) {
eventsChan := stream.Events()
for {
event := <-eventsChan
switch e := event.(type) {
case *types.StartLiveTailResponseStreamMemberSessionStart:
log.Println("Received SessionStart event")
case *types.StartLiveTailResponseStreamMemberSessionUpdate:
for _, logEvent := range e.Value.SessionResults {
log.Println(*logEvent.Message)
}
default:
// Handle on-stream exceptions
if err := stream.Err(); err != nil {
log.Fatalf("Error occured during streaming: %v", err)
} else if event == nil {
log.Println("Stream is Closed")
return
} else {
log.Fatalf("Unknown event type: %T", e)
}
}
}
}
```
Live Tail 세션을 시작합니다.
```
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
panic("configuration error, " + err.Error())
}
client := cloudwatchlogs.NewFromConfig(cfg)
request := &cloudwatchlogs.StartLiveTailInput{
LogGroupIdentifiers: logGroupIdentifiers,
LogStreamNames: logStreamNames,
LogEventFilterPattern: logEventFilterPattern,
}
response, err := client.StartLiveTail(context.TODO(), request)
// Handle pre-stream Exceptions
if err != nil {
log.Fatalf("Failed to start streaming: %v", err)
}
// Start a Goroutine to handle events over stream
stream := response.GetStream()
go handleEventStreamAsync(stream)
```
일정 시간이 경과하면 Live Tail 세션을 중단합니다.
```
// Close the stream (which ends the session) after a timeout
time.Sleep(10 * time.Second)
stream.Close()
log.Println("Event stream closed")
```
+ API 세부 정보는 **AWS SDK for Go API 참조의 [StartLiveTail](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs#Client.StartLiveTail)을 참조하세요.
------
#### [ Java ]
**SDK for Java 2.x**
필수 파일을 포함합니다.
```
import io.reactivex.FlowableSubscriber;
import io.reactivex.annotations.NonNull;
import org.reactivestreams.Subscription;
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.services.cloudwatchlogs.CloudWatchLogsAsyncClient;
import software.amazon.awssdk.services.cloudwatchlogs.model.LiveTailSessionLogEvent;
import software.amazon.awssdk.services.cloudwatchlogs.model.LiveTailSessionStart;
import software.amazon.awssdk.services.cloudwatchlogs.model.LiveTailSessionUpdate;
import software.amazon.awssdk.services.cloudwatchlogs.model.StartLiveTailRequest;
import software.amazon.awssdk.services.cloudwatchlogs.model.StartLiveTailResponseHandler;
import software.amazon.awssdk.services.cloudwatchlogs.model.CloudWatchLogsException;
import software.amazon.awssdk.services.cloudwatchlogs.model.StartLiveTailResponseStream;
import java.util.Date;
import java.util.List;
import java.util.concurrent.atomic.AtomicReference;
```
Live Tail 세션의 이벤트를 처리합니다.
```
private static StartLiveTailResponseHandler getStartLiveTailResponseStreamHandler(
AtomicReference subscriptionAtomicReference) {
return StartLiveTailResponseHandler.builder()
.onResponse(r -> System.out.println("Received initial response"))
.onError(throwable -> {
CloudWatchLogsException e = (CloudWatchLogsException) throwable.getCause();
System.err.println(e.awsErrorDetails().errorMessage());
System.exit(1);
})
.subscriber(() -> new FlowableSubscriber<>() {
@Override
public void onSubscribe(@NonNull Subscription s) {
subscriptionAtomicReference.set(s);
s.request(Long.MAX_VALUE);
}
@Override
public void onNext(StartLiveTailResponseStream event) {
if (event instanceof LiveTailSessionStart) {
LiveTailSessionStart sessionStart = (LiveTailSessionStart) event;
System.out.println(sessionStart);
} else if (event instanceof LiveTailSessionUpdate) {
LiveTailSessionUpdate sessionUpdate = (LiveTailSessionUpdate) event;
List logEvents = sessionUpdate.sessionResults();
logEvents.forEach(e -> {
long timestamp = e.timestamp();
Date date = new Date(timestamp);
System.out.println("[" + date + "] " + e.message());
});
} else {
throw CloudWatchLogsException.builder().message("Unknown event type").build();
}
}
@Override
public void onError(Throwable throwable) {
System.out.println(throwable.getMessage());
System.exit(1);
}
@Override
public void onComplete() {
System.out.println("Completed Streaming Session");
}
})
.build();
}
```
Live Tail 세션을 시작합니다.
```
CloudWatchLogsAsyncClient cloudWatchLogsAsyncClient =
CloudWatchLogsAsyncClient.builder()
.credentialsProvider(ProfileCredentialsProvider.create())
.build();
StartLiveTailRequest request =
StartLiveTailRequest.builder()
.logGroupIdentifiers(logGroupIdentifiers)
.logStreamNames(logStreamNames)
.logEventFilterPattern(logEventFilterPattern)
.build();
/* Create a reference to store the subscription */
final AtomicReference subscriptionAtomicReference = new AtomicReference<>(null);
cloudWatchLogsAsyncClient.startLiveTail(request, getStartLiveTailResponseStreamHandler(subscriptionAtomicReference));
```
일정 시간이 경과하면 Live Tail 세션을 중단합니다.
```
/* Set a timeout for the session and cancel the subscription. This will:
* 1). Close the stream
* 2). Stop the Live Tail session
*/
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
if (subscriptionAtomicReference.get() != null) {
subscriptionAtomicReference.get().cancel();
System.out.println("Subscription to stream closed");
}
```
+ API 세부 정보는 **AWS SDK for Java 2.x API 참조의 [StartLiveTail](https://docs.aws.amazon.com/goto/SdkForJavaV2/logs-2014-03-28/StartLiveTail)을 참조하세요.
------
#### [ JavaScript ]
**SDK for JavaScript(v3)**
필수 파일을 포함합니다.
```
import { CloudWatchLogsClient, StartLiveTailCommand } from "@aws-sdk/client-cloudwatch-logs";
```
Live Tail 세션의 이벤트를 처리합니다.
```
async function handleResponseAsync(response) {
try {
for await (const event of response.responseStream) {
if (event.sessionStart !== undefined) {
console.log(event.sessionStart);
} else if (event.sessionUpdate !== undefined) {
for (const logEvent of event.sessionUpdate.sessionResults) {
const timestamp = logEvent.timestamp;
const date = new Date(timestamp);
console.log("[" + date + "] " + logEvent.message);
}
} else {
console.error("Unknown event type");
}
}
} catch (err) {
// On-stream exceptions are captured here
console.error(err)
}
}
```
Live Tail 세션을 시작합니다.
```
const client = new CloudWatchLogsClient();
const command = new StartLiveTailCommand({
logGroupIdentifiers: logGroupIdentifiers,
logStreamNames: logStreamNames,
logEventFilterPattern: filterPattern
});
try{
const response = await client.send(command);
handleResponseAsync(response);
} catch (err){
// Pre-stream exceptions are captured here
console.log(err);
}
```
일정 시간이 경과하면 Live Tail 세션을 중단합니다.
```
/* Set a timeout to close the client. This will stop the Live Tail session. */
setTimeout(function() {
console.log("Client timeout");
client.destroy();
}, 10000);
```
+ API 세부 정보는 **AWS SDK for JavaScript API 참조의 [StartLiveTail](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/StartLiveTailCommand)을 참조하세요.
------
#### [ Kotlin ]
**SDK for Kotlin**
필수 파일을 포함합니다.
```
import aws.sdk.kotlin.services.cloudwatchlogs.CloudWatchLogsClient
import aws.sdk.kotlin.services.cloudwatchlogs.model.StartLiveTailRequest
import aws.sdk.kotlin.services.cloudwatchlogs.model.StartLiveTailResponseStream
import kotlinx.coroutines.flow.takeWhile
```
Live Tail 세션을 시작합니다.
```
val client = CloudWatchLogsClient.fromEnvironment()
val request = StartLiveTailRequest {
logGroupIdentifiers = logGroupIdentifiersVal
logStreamNames = logStreamNamesVal
logEventFilterPattern = logEventFilterPatternVal
}
val startTime = System.currentTimeMillis()
try {
client.startLiveTail(request) { response ->
val stream = response.responseStream
if (stream != null) {
/* Set a timeout to unsubcribe from the flow. This will:
* 1). Close the stream
* 2). Stop the Live Tail session
*/
stream.takeWhile { System.currentTimeMillis() - startTime < 10000 }.collect { value ->
if (value is StartLiveTailResponseStream.SessionStart) {
println(value.asSessionStart())
} else if (value is StartLiveTailResponseStream.SessionUpdate) {
for (e in value.asSessionUpdate().sessionResults!!) {
println(e)
}
} else {
throw IllegalArgumentException("Unknown event type")
}
}
} else {
throw IllegalArgumentException("No response stream")
}
}
} catch (e: Exception) {
println("Exception occurred during StartLiveTail: $e")
System.exit(1)
}
```
+ API 세부 정보는 **AWS SDK for Kotlin API 참조의 [StartLiveTail](https://sdk.amazonaws.com/kotlin/api/latest/index.html)을 참조하세요.
------
#### [ Python ]
**SDK for Python(Boto3)**
필수 파일을 포함합니다.
```
import boto3
import time
from datetime import datetime
```
Live Tail 세션을 시작합니다.
```
# Initialize the client
client = boto3.client('logs')
start_time = time.time()
try:
response = client.start_live_tail(
logGroupIdentifiers=log_group_identifiers,
logStreamNames=log_streams,
logEventFilterPattern=filter_pattern
)
event_stream = response['responseStream']
# Handle the events streamed back in the response
for event in event_stream:
# Set a timeout to close the stream.
# This will end the Live Tail session.
if (time.time() - start_time >= 10):
event_stream.close()
break
# Handle when session is started
if 'sessionStart' in event:
session_start_event = event['sessionStart']
print(session_start_event)
# Handle when log event is given in a session update
elif 'sessionUpdate' in event:
log_events = event['sessionUpdate']['sessionResults']
for log_event in log_events:
print('[{date}] {log}'.format(date=datetime.fromtimestamp(log_event['timestamp']/1000),log=log_event['message']))
else:
# On-stream exceptions are captured here
raise RuntimeError(str(event))
except Exception as e:
print(e)
```
+ API 세부 정보는 **AWS SDK for Python(Boto3) API 참조의 [StartLiveTail](https://docs.aws.amazon.com/goto/boto3/logs-2014-03-28/StartLiveTail)을 참조하세요.
------
# AWS SDK와 `StartQuery` 함께 사용
다음 코드 예시는 `StartQuery`의 사용 방법을 보여 줍니다.
작업 예제는 대규모 프로그램에서 발췌한 코드이며 컨텍스트에 맞춰 실행해야 합니다. 다음 코드 예제에서는 컨텍스트 내에서 이 작업을 확인할 수 있습니다.
+ [대용량 쿼리 실행](cloudwatch-logs_example_cloudwatch-logs_Scenario_BigQuery_section.md)
------
#### [ .NET ]
**SDK for .NET (v4)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv4/CloudWatchLogs/LargeQuery#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
///
/// Starts a CloudWatch Logs Insights query.
///
/// The name of the log group to query.
/// The CloudWatch Logs Insights query string.
/// The start time for the query (seconds since epoch).
/// The end time for the query (seconds since epoch).
/// The maximum number of results to return.
/// The query ID if successful, null otherwise.
public async Task StartQueryAsync(
string logGroupName,
string queryString,
long startTime,
long endTime,
int limit = 10000)
{
try
{
var request = new StartQueryRequest
{
LogGroupName = logGroupName,
QueryString = queryString,
StartTime = startTime,
EndTime = endTime,
Limit = limit
};
var response = await _amazonCloudWatchLogs.StartQueryAsync(request);
return response.QueryId;
}
catch (InvalidParameterException ex)
{
_logger.LogError($"Invalid parameter for query: {ex.Message}");
return null;
}
catch (ResourceNotFoundException ex)
{
_logger.LogError($"Log group not found: {ex.Message}");
return null;
}
catch (Exception ex)
{
_logger.LogError($"An error occurred while starting query: {ex.Message}");
return null;
}
}
```
+ API 세부 정보는 *AWS SDK for .NET API 참조*의 [StartQuery](https://docs.aws.amazon.com/goto/DotNetSDKV4/logs-2014-03-28/StartQuery)를 참조하세요.
------
#### [ JavaScript ]
**SDK for JavaScript (v3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
/**
* Wrapper for the StartQueryCommand. Uses a static query string
* for consistency.
* @param {[Date, Date]} dateRange
* @param {number} maxLogs
* @returns {Promise<{ queryId: string }>}
*/
async _startQuery([startDate, endDate], maxLogs = 10000) {
try {
return await this.client.send(
new StartQueryCommand({
logGroupNames: this.logGroupNames,
queryString: "fields @timestamp, @message | sort @timestamp asc",
startTime: startDate.valueOf(),
endTime: endDate.valueOf(),
limit: maxLogs,
}),
);
} catch (err) {
/** @type {string} */
const message = err.message;
if (message.startsWith("Query's end date and time")) {
// This error indicates that the query's start or end date occur
// before the log group was created.
throw new DateOutOfBoundsError(message);
}
throw err;
}
}
```
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 [StartQuery](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/StartQueryCommand)를 참조하세요.
------
#### [ Python ]
**SDK for Python(Boto3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/cloudwatch-logs#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
def perform_query(self, date_range):
"""
Performs the actual CloudWatch log query.
:param date_range: A tuple representing the start and end datetime for the query.
:type date_range: tuple
:return: A list containing the query results.
:rtype: list
"""
client = boto3.client("logs")
try:
try:
start_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[0])
)
end_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[1])
)
response = client.start_query(
logGroupName=self.log_group,
startTime=start_time,
endTime=end_time,
queryString=self.query_string,
limit=self.limit,
)
query_id = response["queryId"]
except client.exceptions.ResourceNotFoundException as e:
raise DateOutOfBoundsError(f"Resource not found: {e}")
while True:
time.sleep(1)
results = client.get_query_results(queryId=query_id)
if results["status"] in [
"Complete",
"Failed",
"Cancelled",
"Timeout",
"Unknown",
]:
return results.get("results", [])
except DateOutOfBoundsError:
return []
def _initiate_query(self, client, date_range, max_logs):
"""
Initiates the CloudWatch logs query.
:param date_range: A tuple representing the start and end datetime for the query.
:type date_range: tuple
:param max_logs: The maximum number of logs to retrieve.
:type max_logs: int
:return: The query ID as a string.
:rtype: str
"""
try:
start_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[0])
)
end_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[1])
)
response = client.start_query(
logGroupName=self.log_group,
startTime=start_time,
endTime=end_time,
queryString=self.query_string,
limit=max_logs,
)
return response["queryId"]
except client.exceptions.ResourceNotFoundException as e:
raise DateOutOfBoundsError(f"Resource not found: {e}")
```
+ API 세부 정보는 *AWS SDK for Python(Boto3) API 참조*의 [StartQuery](https://docs.aws.amazon.com/goto/boto3/logs-2014-03-28/StartQuery)를 참조하세요.
------
#### [ SAP ABAP ]
**SDK for SAP ABAP API**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/sap-abap/services/cwl#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
```
TRY.
" iv_log_group_name = '/aws/lambda/my-function'
" iv_query_string = 'fields @timestamp, @message | sort @timestamp desc | limit 20'
" iv_start_time and iv_end_time must be in Unix epoch milliseconds (ms since Jan 1, 1970 00:00:00 UTC)
oo_result = lo_cwl->startquery(
iv_loggroupname = iv_log_group_name
iv_starttime = iv_start_time
iv_endtime = iv_end_time
iv_querystring = iv_query_string
iv_limit = iv_limit ).
" Display the query ID for tracking
DATA(lv_query_id) = oo_result->get_queryid( ).
MESSAGE |Query started successfully with ID: { lv_query_id }| TYPE 'I'.
CATCH /aws1/cx_cwlinvalidparameterex.
MESSAGE 'Invalid parameter.' TYPE 'E'.
CATCH /aws1/cx_cwllimitexceededex.
MESSAGE 'Limit exceeded.' TYPE 'E'.
CATCH /aws1/cx_cwlmalformedqueryex.
MESSAGE 'Malformed query.' TYPE 'E'.
CATCH /aws1/cx_cwlresourcenotfoundex.
MESSAGE 'Resource not found.' TYPE 'E'.
CATCH /aws1/cx_cwlserviceunavailex.
MESSAGE 'Service unavailable.' TYPE 'E'.
ENDTRY.
```
+ API 세부 정보는 *AWS SDK for SAP ABAP API 참조*의 [StartQuery](https://docs.aws.amazon.com/sdk-for-sap-abap/v1/api/latest/index.html)를 참조하세요.
------
# AWS SDKs를 사용한 CloudWatch Logs 시나리오
다음 코드 예제에서는 CloudWatch Logs AWS SDKs에서 일반적인 시나리오를 구현하는 방법을 보여줍니다. 이러한 시나리오는 CloudWatch Logs 내에서 또는 다른 AWS 서비스와 결합된 상태에서 여러 함수를 직접 호출하여 특정 작업을 수행하는 방법을 보여줍니다. 각 시나리오에는 전체 소스 코드에 대한 링크가 포함되어 있습니다. 여기에서 코드를 설정 및 실행하는 방법에 대한 지침을 찾을 수 있습니다.
시나리오는 컨텍스트에 맞는 서비스 작업을 이해하는 데 도움이 되도록 중급 수준의 경험을 대상으로 합니다.
**Topics**
+ [대용량 쿼리 실행](cloudwatch-logs_example_cloudwatch-logs_Scenario_BigQuery_section.md)
+ [예약된 이벤트를 사용하여 Lambda 함수 간접 호출](cloudwatch-logs_example_cross_LambdaScheduledEvents_section.md)
# CloudWatch Logs를 사용하여 대규모 쿼리 실행
다음 코드 예제는 CloudWatch Logs를 사용하여 10,000개 이상의 레코드를 쿼리하는 방법을 보여줍니다.
------
#### [ .NET ]
**SDK for .NET (v4)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/dotnetv4/CloudWatchLogs/LargeQuery#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
이는 대규모 쿼리 시나리오를 보여주는 기본 워크플로입니다.
```
using System.Diagnostics;
using System.Text.RegularExpressions;
using Amazon.CloudFormation;
using Amazon.CloudFormation.Model;
using Amazon.CloudWatchLogs;
using Amazon.CloudWatchLogs.Model;
using CloudWatchLogsActions;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
namespace CloudWatchLogsScenario;
public class LargeQueryWorkflow
{
/*
Before running this .NET code example, set up your development environment, including your credentials.
This .NET code example performs the following tasks for the CloudWatch Logs Large Query workflow:
1. Prepare the Application:
- Prompt the user to deploy CloudFormation stack and generate sample logs.
- Deploy the CloudFormation template for resource creation.
- Generate 50,000 sample log entries using CloudWatch Logs API.
- Wait 5 minutes for logs to be fully ingested.
2. Execute Large Query:
- Perform recursive queries to retrieve all logs using binary search.
- Display progress for each query executed.
- Show total execution time and logs found.
3. Clean up:
- Prompt the user to delete the CloudFormation stack and all resources.
- Destroy the CloudFormation stack and wait until removed.
*/
public static ILogger _logger = null!;
public static CloudWatchLogsWrapper _wrapper = null!;
public static IAmazonCloudFormation _amazonCloudFormation = null!;
private static string _logGroupName = "/workflows/cloudwatch-logs/large-query";
private static string _logStreamName = "stream1";
private static long _queryStartDate;
private static long _queryEndDate;
public static bool _interactive = true;
public static string _stackName = "CloudWatchLargeQueryStack";
private static string _stackResourcePath = "../../../../../../../scenarios/features/cloudwatch_logs_large_query/resources/stack.yaml";
public static async Task Main(string[] args)
{
using var host = Host.CreateDefaultBuilder(args)
.ConfigureLogging(logging =>
logging.AddFilter("System", LogLevel.Debug)
.AddFilter("Microsoft", LogLevel.Information))
.ConfigureServices((_, services) =>
services.AddAWSService()
.AddAWSService()
.AddTransient()
)
.Build();
if (_interactive)
{
_logger = LoggerFactory.Create(builder => { builder.AddConsole(); })
.CreateLogger();
_wrapper = host.Services.GetRequiredService();
_amazonCloudFormation = host.Services.GetRequiredService();
}
Console.WriteLine(new string('-', 80));
Console.WriteLine("Welcome to the CloudWatch Logs Large Query Scenario.");
Console.WriteLine(new string('-', 80));
Console.WriteLine("This scenario demonstrates how to perform large-scale queries on");
Console.WriteLine("CloudWatch Logs using recursive binary search to retrieve more than");
Console.WriteLine("the 10,000 result limit.");
Console.WriteLine();
try
{
Console.WriteLine(new string('-', 80));
var prepareSuccess = await PrepareApplication();
Console.WriteLine(new string('-', 80));
if (prepareSuccess)
{
Console.WriteLine(new string('-', 80));
await ExecuteLargeQuery();
Console.WriteLine(new string('-', 80));
}
Console.WriteLine(new string('-', 80));
await Cleanup();
Console.WriteLine(new string('-', 80));
}
catch (Exception ex)
{
_logger.LogError(ex, "There was a problem with the scenario, initiating cleanup...");
_interactive = false;
await Cleanup();
}
Console.WriteLine("CloudWatch Logs Large Query scenario completed.");
}
///
/// Runs the scenario workflow. Used for testing.
///
public static async Task RunScenario()
{
Console.WriteLine(new string('-', 80));
Console.WriteLine("Welcome to the CloudWatch Logs Large Query Scenario.");
Console.WriteLine(new string('-', 80));
Console.WriteLine("This scenario demonstrates how to perform large-scale queries on");
Console.WriteLine("CloudWatch Logs using recursive binary search to retrieve more than");
Console.WriteLine("the 10,000 result limit.");
Console.WriteLine();
try
{
Console.WriteLine(new string('-', 80));
var prepareSuccess = await PrepareApplication();
Console.WriteLine(new string('-', 80));
if (prepareSuccess)
{
Console.WriteLine(new string('-', 80));
await ExecuteLargeQuery();
Console.WriteLine(new string('-', 80));
}
Console.WriteLine(new string('-', 80));
await Cleanup();
Console.WriteLine(new string('-', 80));
}
catch (Exception ex)
{
_logger.LogError(ex, "There was a problem with the scenario, initiating cleanup...");
_interactive = false;
await Cleanup();
}
Console.WriteLine("CloudWatch Logs Large Query scenario completed.");
}
///
/// Prepares the application by creating the necessary resources.
///
/// True if the application was prepared successfully.
public static async Task PrepareApplication()
{
Console.WriteLine("Preparing the application...");
Console.WriteLine();
try
{
var deployStack = !_interactive || GetYesNoResponse(
"Would you like to deploy the CloudFormation stack and generate sample logs? (y/n) ");
if (deployStack)
{
if (_interactive)
{
Console.Write(
$"Enter a path for the CloudFormation stack resource .yaml file (or press Enter for default '{_stackResourcePath}'): ");
string? inputPath = Console.ReadLine();
if (!string.IsNullOrWhiteSpace(inputPath))
{
_stackResourcePath = inputPath;
}
}
_stackName = PromptUserForStackName();
var deploySuccess = await DeployCloudFormationStack(_stackName);
if (deploySuccess)
{
Console.WriteLine();
Console.WriteLine("Generating 50,000 sample log entries...");
var generateSuccess = await GenerateSampleLogs();
if (generateSuccess)
{
Console.WriteLine();
Console.WriteLine("Sample logs created. Waiting 5 minutes for logs to be fully ingested...");
await WaitWithCountdown(300);
Console.WriteLine("Application preparation complete.");
return true;
}
}
}
else
{
_logGroupName = PromptUserForInput("Enter the log group name ", _logGroupName);
_logStreamName = PromptUserForInput("Enter the log stream name ", _logStreamName);
var startDateMs = PromptUserForLong("Enter the query start date (milliseconds since epoch): ");
var endDateMs = PromptUserForLong("Enter the query end date (milliseconds since epoch): ");
_queryStartDate = startDateMs / 1000;
_queryEndDate = endDateMs / 1000;
Console.WriteLine("Application preparation complete.");
return true;
}
}
catch (Exception ex)
{
_logger.LogError(ex, "An error occurred while preparing the application.");
}
Console.WriteLine("Application preparation failed.");
return false;
}
///
/// Deploys the CloudFormation stack with the necessary resources.
///
/// The name of the CloudFormation stack.
/// True if the stack was deployed successfully.
private static async Task DeployCloudFormationStack(string stackName)
{
Console.WriteLine($"\nDeploying CloudFormation stack: {stackName}");
try
{
var request = new CreateStackRequest
{
StackName = stackName,
TemplateBody = await File.ReadAllTextAsync(_stackResourcePath)
};
var response = await _amazonCloudFormation.CreateStackAsync(request);
if (response.HttpStatusCode == System.Net.HttpStatusCode.OK)
{
Console.WriteLine($"CloudFormation stack creation started: {stackName}");
bool stackCreated = await WaitForStackCompletion(response.StackId);
if (stackCreated)
{
Console.WriteLine("CloudFormation stack created successfully.");
return true;
}
else
{
_logger.LogError($"CloudFormation stack creation failed: {stackName}");
return false;
}
}
else
{
_logger.LogError($"Failed to create CloudFormation stack: {stackName}");
return false;
}
}
catch (AlreadyExistsException)
{
_logger.LogWarning($"CloudFormation stack '{stackName}' already exists. Please provide a unique name.");
var newStackName = PromptUserForStackName();
return await DeployCloudFormationStack(newStackName);
}
catch (Exception ex)
{
_logger.LogError(ex, $"An error occurred while deploying the CloudFormation stack: {stackName}");
return false;
}
}
///
/// Waits for the CloudFormation stack to be in the CREATE_COMPLETE state.
///
/// The ID of the CloudFormation stack.
/// True if the stack was created successfully.
private static async Task WaitForStackCompletion(string stackId)
{
int retryCount = 0;
const int maxRetries = 30;
const int retryDelay = 10000;
while (retryCount < maxRetries)
{
var describeStacksRequest = new DescribeStacksRequest
{
StackName = stackId
};
var describeStacksResponse = await _amazonCloudFormation.DescribeStacksAsync(describeStacksRequest);
if (describeStacksResponse.Stacks.Count > 0)
{
if (describeStacksResponse.Stacks[0].StackStatus == StackStatus.CREATE_COMPLETE)
{
return true;
}
if (describeStacksResponse.Stacks[0].StackStatus == StackStatus.CREATE_FAILED ||
describeStacksResponse.Stacks[0].StackStatus == StackStatus.ROLLBACK_COMPLETE)
{
return false;
}
}
Console.WriteLine("Waiting for CloudFormation stack creation to complete...");
await Task.Delay(retryDelay);
retryCount++;
}
_logger.LogError("Timed out waiting for CloudFormation stack creation to complete.");
return false;
}
///
/// Generates sample logs directly using CloudWatch Logs API.
/// Creates 50,000 log entries spanning 5 minutes.
///
/// True if logs were generated successfully.
private static async Task GenerateSampleLogs()
{
const int totalEntries = 50000;
const int entriesPerBatch = 10000;
const int fiveMinutesMs = 5 * 60 * 1000;
try
{
// Calculate timestamps
var startTimeMs = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();
var timestampIncrement = fiveMinutesMs / totalEntries;
Console.WriteLine($"Generating {totalEntries} log entries...");
var entryCount = 0;
var currentTimestamp = startTimeMs;
var numBatches = totalEntries / entriesPerBatch;
// Generate and upload logs in batches
for (int batchNum = 0; batchNum < numBatches; batchNum++)
{
var logEvents = new List();
for (int i = 0; i < entriesPerBatch; i++)
{
logEvents.Add(new InputLogEvent
{
Timestamp = DateTimeOffset.FromUnixTimeMilliseconds(currentTimestamp).UtcDateTime,
Message = $"Entry {entryCount}"
});
entryCount++;
currentTimestamp += timestampIncrement;
}
// Upload batch
var success = await _wrapper.PutLogEventsAsync(_logGroupName, _logStreamName, logEvents);
if (!success)
{
_logger.LogError($"Failed to upload batch {batchNum + 1}/{numBatches}");
return false;
}
Console.WriteLine($"Uploaded batch {batchNum + 1}/{numBatches}");
}
// Set query date range (convert milliseconds to seconds for query API)
_queryStartDate = startTimeMs / 1000;
_queryEndDate = (currentTimestamp - timestampIncrement) / 1000;
Console.WriteLine($"Query start date: {DateTimeOffset.FromUnixTimeSeconds(_queryStartDate):yyyy-MM-ddTHH:mm:ss.fffZ}");
Console.WriteLine($"Query end date: {DateTimeOffset.FromUnixTimeSeconds(_queryEndDate):yyyy-MM-ddTHH:mm:ss.fffZ}");
Console.WriteLine($"Successfully uploaded {totalEntries} log entries");
return true;
}
catch (Exception ex)
{
_logger.LogError(ex, "An error occurred while generating sample logs.");
return false;
}
}
///
/// Executes the large query workflow.
///
public static async Task ExecuteLargeQuery()
{
Console.WriteLine("Starting recursive query to retrieve all logs...");
Console.WriteLine();
var queryLimit = PromptUserForInteger("Enter the query limit (max 10000) ", 10000);
if (queryLimit > 10000) queryLimit = 10000;
var queryString = "fields @timestamp, @message | sort @timestamp asc";
var stopwatch = Stopwatch.StartNew();
var allResults = await PerformLargeQuery(_logGroupName, queryString, _queryStartDate, _queryEndDate, queryLimit);
stopwatch.Stop();
Console.WriteLine();
Console.WriteLine($"Queries finished in {stopwatch.Elapsed.TotalSeconds:F3} seconds.");
Console.WriteLine($"Total logs found: {allResults.Count}");
// Check for duplicates
Console.WriteLine();
Console.WriteLine("Checking for duplicate logs...");
var duplicates = FindDuplicateLogs(allResults);
if (duplicates.Count > 0)
{
Console.WriteLine($"WARNING: Found {duplicates.Count} duplicate log entries!");
Console.WriteLine("Duplicate entries (showing first 10):");
foreach (var dup in duplicates.Take(10))
{
Console.WriteLine($" [{dup.Timestamp}] {dup.Message} (appears {dup.Count} times)");
}
var uniqueCount = allResults.Count - duplicates.Sum(d => d.Count - 1);
Console.WriteLine($"Unique logs: {uniqueCount}");
}
else
{
Console.WriteLine("No duplicates found. All logs are unique.");
}
Console.WriteLine();
var viewSample = !_interactive || GetYesNoResponse("Would you like to see a sample of the logs? (y/n) ");
if (viewSample)
{
Console.WriteLine();
Console.WriteLine($"Sample logs (first 10 of {allResults.Count}):");
for (int i = 0; i < Math.Min(10, allResults.Count); i++)
{
var timestamp = allResults[i].Find(f => f.Field == "@timestamp")?.Value ?? "N/A";
var message = allResults[i].Find(f => f.Field == "@message")?.Value ?? "N/A";
Console.WriteLine($"[{timestamp}] {message}");
}
}
}
///
/// Performs a large query using recursive binary search.
///
private static async Task>> PerformLargeQuery(
string logGroupName,
string queryString,
long startTime,
long endTime,
int limit)
{
var queryId = await _wrapper.StartQueryAsync(logGroupName, queryString, startTime, endTime, limit);
if (queryId == null)
{
return new List>();
}
var results = await PollQueryResults(queryId);
if (results == null || results.Count == 0)
{
return new List>();
}
var startDate = DateTimeOffset.FromUnixTimeSeconds(startTime).ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
var endDate = DateTimeOffset.FromUnixTimeSeconds(endTime).ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
Console.WriteLine($"Query date range: {startDate} ({startTime}s) to {endDate} ({endTime}s). Found {results.Count} logs.");
if (results.Count < limit)
{
Console.WriteLine($" -> Returning {results.Count} logs (less than limit of {limit})");
return results;
}
Console.WriteLine($" -> Hit limit of {limit}. Need to split and recurse.");
// Get the timestamp of the last log (sorted to find the actual last one)
var lastLogTimestamp = GetLastLogTimestamp(results);
if (lastLogTimestamp == null)
{
Console.WriteLine($" -> No timestamp found in results. Returning {results.Count} logs.");
return results;
}
Console.WriteLine($" -> Last log timestamp: {lastLogTimestamp}");
// Parse the timestamp and add 1 millisecond to avoid querying the same log again
var lastLogDate = DateTimeOffset.Parse(lastLogTimestamp + " +0000");
Console.WriteLine($" -> Last log as DateTimeOffset: {lastLogDate:yyyy-MM-ddTHH:mm:ss.fffZ} ({lastLogDate.ToUnixTimeSeconds()}s)");
var offsetLastLogDate = lastLogDate.AddMilliseconds(1);
Console.WriteLine($" -> Offset timestamp (last + 1ms): {offsetLastLogDate:yyyy-MM-ddTHH:mm:ss.fffZ} ({offsetLastLogDate.ToUnixTimeSeconds()}s)");
// Convert to seconds, but round UP to the next second to avoid overlapping with logs in the same second
// This ensures we don't re-query logs that share the same second as the last log
var offsetLastLogTime = offsetLastLogDate.ToUnixTimeSeconds();
if (offsetLastLogDate.Millisecond > 0)
{
offsetLastLogTime++; // Move to the next full second
Console.WriteLine($" -> Adjusted to next full second: {offsetLastLogTime}s ({DateTimeOffset.FromUnixTimeSeconds(offsetLastLogTime):yyyy-MM-ddTHH:mm:ss.fffZ})");
}
Console.WriteLine($" -> Comparing: offsetLastLogTime={offsetLastLogTime}s vs endTime={endTime}s");
Console.WriteLine($" -> End time as date: {DateTimeOffset.FromUnixTimeSeconds(endTime):yyyy-MM-ddTHH:mm:ss.fffZ}");
// Check if there's any time range left to query
if (offsetLastLogTime >= endTime)
{
Console.WriteLine($" -> No time range left to query. Offset time ({offsetLastLogTime}s) >= end time ({endTime}s)");
return results;
}
// Split the remaining date range in half
var (range1Start, range1End, range2Start, range2End) = SplitDateRange(offsetLastLogTime, endTime);
var range1StartDate = DateTimeOffset.FromUnixTimeSeconds(range1Start).ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
var range1EndDate = DateTimeOffset.FromUnixTimeSeconds(range1End).ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
var range2StartDate = DateTimeOffset.FromUnixTimeSeconds(range2Start).ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
var range2EndDate = DateTimeOffset.FromUnixTimeSeconds(range2End).ToString("yyyy-MM-ddTHH:mm:ss.fffZ");
Console.WriteLine($" -> Splitting remaining range:");
Console.WriteLine($" Range 1: {range1StartDate} ({range1Start}s) to {range1EndDate} ({range1End}s)");
Console.WriteLine($" Range 2: {range2StartDate} ({range2Start}s) to {range2EndDate} ({range2End}s)");
// Query both halves recursively
Console.WriteLine($" -> Querying range 1...");
var results1 = await PerformLargeQuery(logGroupName, queryString, range1Start, range1End, limit);
Console.WriteLine($" -> Range 1 returned {results1.Count} logs");
Console.WriteLine($" -> Querying range 2...");
var results2 = await PerformLargeQuery(logGroupName, queryString, range2Start, range2End, limit);
Console.WriteLine($" -> Range 2 returned {results2.Count} logs");
// Combine all results
var allResults = new List>(results);
allResults.AddRange(results1);
allResults.AddRange(results2);
Console.WriteLine($" -> Combined total: {allResults.Count} logs ({results.Count} + {results1.Count} + {results2.Count})");
return allResults;
}
///
/// Gets the timestamp string of the most recent log from a list of logs.
/// Sorts timestamps to find the actual last one.
///
private static string? GetLastLogTimestamp(List> logs)
{
var timestamps = logs
.Select(log => log.Find(f => f.Field == "@timestamp")?.Value)
.Where(t => !string.IsNullOrEmpty(t))
.OrderBy(t => t)
.ToList();
if (timestamps.Count == 0)
{
return null;
}
return timestamps[timestamps.Count - 1];
}
///
/// Splits a date range in half.
/// Range 2 starts at midpoint + 1 second to avoid overlap.
///
private static (long range1Start, long range1End, long range2Start, long range2End) SplitDateRange(long startTime, long endTime)
{
var midpoint = startTime + (endTime - startTime) / 2;
// Range 2 starts at midpoint + 1 to avoid querying the same second twice
return (startTime, midpoint, midpoint + 1, endTime);
}
///
/// Polls for query results until complete.
///
private static async Task>?> PollQueryResults(string queryId)
{
int retryCount = 0;
const int maxRetries = 60;
const int retryDelay = 1000;
while (retryCount < maxRetries)
{
var response = await _wrapper.GetQueryResultsAsync(queryId);
if (response == null)
{
return null;
}
if (response.Status == QueryStatus.Complete)
{
return response.Results;
}
if (response.Status == QueryStatus.Failed ||
response.Status == QueryStatus.Cancelled ||
response.Status == QueryStatus.Timeout ||
response.Status == QueryStatus.Unknown)
{
_logger.LogError($"Query failed with status: {response.Status}");
return null;
}
await Task.Delay(retryDelay);
retryCount++;
}
_logger.LogError("Timed out waiting for query results.");
return null;
}
///
/// Cleans up the resources created during the scenario.
///
public static async Task Cleanup()
{
var cleanup = !_interactive || GetYesNoResponse(
"Do you want to delete the CloudFormation stack and all resources? (y/n) ");
if (cleanup)
{
try
{
var stackDeleteSuccess = await DeleteCloudFormationStack(_stackName, false);
return stackDeleteSuccess;
}
catch (Exception ex)
{
_logger.LogError(ex, "An error occurred while cleaning up the resources.");
return false;
}
}
Console.WriteLine($"Resources will remain. Stack name: {_stackName}, Log group: {_logGroupName}");
_logger.LogInformation("CloudWatch Logs Large Query scenario is complete.");
return true;
}
///
/// Deletes the CloudFormation stack and waits for confirmation.
///
private static async Task DeleteCloudFormationStack(string stackName, bool forceDelete)
{
var request = new DeleteStackRequest
{
StackName = stackName,
};
if (forceDelete)
{
request.DeletionMode = DeletionMode.FORCE_DELETE_STACK;
}
await _amazonCloudFormation.DeleteStackAsync(request);
Console.WriteLine($"CloudFormation stack '{stackName}' is being deleted. This may take a few minutes.");
bool stackDeleted = await WaitForStackDeletion(stackName, forceDelete);
if (stackDeleted)
{
Console.WriteLine($"CloudFormation stack '{stackName}' has been deleted.");
return true;
}
else
{
_logger.LogError($"Failed to delete CloudFormation stack '{stackName}'.");
return false;
}
}
///
/// Waits for the stack to be deleted.
///
private static async Task WaitForStackDeletion(string stackName, bool forceDelete)
{
int retryCount = 0;
const int maxRetries = 30;
const int retryDelay = 10000;
while (retryCount < maxRetries)
{
var describeStacksRequest = new DescribeStacksRequest
{
StackName = stackName
};
try
{
var describeStacksResponse = await _amazonCloudFormation.DescribeStacksAsync(describeStacksRequest);
if (describeStacksResponse.Stacks.Count == 0 ||
describeStacksResponse.Stacks[0].StackStatus == StackStatus.DELETE_COMPLETE)
{
return true;
}
if (!forceDelete && describeStacksResponse.Stacks[0].StackStatus == StackStatus.DELETE_FAILED)
{
return await DeleteCloudFormationStack(stackName, true);
}
}
catch (AmazonCloudFormationException ex) when (ex.ErrorCode == "ValidationError")
{
return true;
}
Console.WriteLine($"Waiting for CloudFormation stack '{stackName}' to be deleted...");
await Task.Delay(retryDelay);
retryCount++;
}
_logger.LogError($"Timed out waiting for CloudFormation stack '{stackName}' to be deleted.");
return false;
}
///
/// Waits with a countdown display.
///
private static async Task WaitWithCountdown(int seconds)
{
for (int i = seconds; i > 0; i--)
{
Console.Write($"\rWaiting: {i} seconds remaining... ");
await Task.Delay(1000);
}
Console.WriteLine("\rWait complete. ");
}
///
/// Helper method to get a yes or no response from the user.
///
private static bool GetYesNoResponse(string question)
{
Console.WriteLine(question);
var ynResponse = Console.ReadLine();
var response = ynResponse != null && ynResponse.Equals("y", StringComparison.InvariantCultureIgnoreCase);
return response;
}
///
/// Prompts the user for a stack name.
///
private static string PromptUserForStackName()
{
if (_interactive)
{
Console.Write($"Enter a name for the CloudFormation stack (press Enter for default '{_stackName}'): ");
string? input = Console.ReadLine();
if (!string.IsNullOrWhiteSpace(input))
{
var regex = "[a-zA-Z][-a-zA-Z0-9]*";
if (!Regex.IsMatch(input, regex))
{
Console.WriteLine($"Invalid stack name. Using default: {_stackName}");
return _stackName;
}
return input;
}
}
return _stackName;
}
///
/// Prompts the user for input with a default value.
///
private static string PromptUserForInput(string prompt, string defaultValue)
{
if (_interactive)
{
Console.Write($"{prompt}(press Enter for default '{defaultValue}'): ");
string? input = Console.ReadLine();
return string.IsNullOrWhiteSpace(input) ? defaultValue : input;
}
return defaultValue;
}
///
/// Prompts the user for an integer value.
///
private static int PromptUserForInteger(string prompt, int defaultValue)
{
if (_interactive)
{
Console.Write($"{prompt}(press Enter for default '{defaultValue}'): ");
string? input = Console.ReadLine();
if (string.IsNullOrWhiteSpace(input) || !int.TryParse(input, out var result))
{
return defaultValue;
}
return result;
}
return defaultValue;
}
///
/// Prompts the user for a long value.
///
private static long PromptUserForLong(string prompt)
{
if (_interactive)
{
Console.Write(prompt);
string? input = Console.ReadLine();
if (long.TryParse(input, out var result))
{
return result;
}
}
return 0;
}
///
/// Finds duplicate log entries based on timestamp and message.
///
private static List<(string Timestamp, string Message, int Count)> FindDuplicateLogs(List> logs)
{
var logSignatures = new Dictionary();
foreach (var log in logs)
{
var timestamp = log.Find(f => f.Field == "@timestamp")?.Value ?? "";
var message = log.Find(f => f.Field == "@message")?.Value ?? "";
var signature = $"{timestamp}|{message}";
if (logSignatures.ContainsKey(signature))
{
logSignatures[signature]++;
}
else
{
logSignatures[signature] = 1;
}
}
return logSignatures
.Where(kvp => kvp.Value > 1)
.Select(kvp =>
{
var parts = kvp.Key.Split('|');
return (Timestamp: parts[0], Message: parts[1], Count: kvp.Value);
})
.OrderByDescending(x => x.Count)
.ToList();
}
}
```
+ API 세부 정보는 *AWS SDK for .NET API 참조*의 다음 주제를 참조하세요.
+ [GetQueryResults](https://docs.aws.amazon.com/goto/DotNetSDKV4/logs-2014-03-28/GetQueryResults)
+ [StartQuery](https://docs.aws.amazon.com/goto/DotNetSDKV4/logs-2014-03-28/StartQuery)
------
#### [ JavaScript ]
**SDK for JavaScript (v3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cloudwatch-logs/scenarios/large-query#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
진입점입니다.
```
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
import { CloudWatchLogsClient } from "@aws-sdk/client-cloudwatch-logs";
import { CloudWatchQuery } from "./cloud-watch-query.js";
console.log("Starting a recursive query...");
if (!process.env.QUERY_START_DATE || !process.env.QUERY_END_DATE) {
throw new Error(
"QUERY_START_DATE and QUERY_END_DATE environment variables are required.",
);
}
const cloudWatchQuery = new CloudWatchQuery(new CloudWatchLogsClient({}), {
logGroupNames: ["/workflows/cloudwatch-logs/large-query"],
dateRange: [
new Date(Number.parseInt(process.env.QUERY_START_DATE)),
new Date(Number.parseInt(process.env.QUERY_END_DATE)),
],
});
await cloudWatchQuery.run();
console.log(
`Queries finished in ${cloudWatchQuery.secondsElapsed} seconds.\nTotal logs found: ${cloudWatchQuery.results.length}`,
);
```
필요한 경우 쿼리를 여러 단계로 분할하는 클래스입니다.
```
// Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
// SPDX-License-Identifier: Apache-2.0
import {
StartQueryCommand,
GetQueryResultsCommand,
} from "@aws-sdk/client-cloudwatch-logs";
import { splitDateRange } from "@aws-doc-sdk-examples/lib/utils/util-date.js";
import { retry } from "@aws-doc-sdk-examples/lib/utils/util-timers.js";
class DateOutOfBoundsError extends Error {}
export class CloudWatchQuery {
/**
* Run a query for all CloudWatch Logs within a certain date range.
* CloudWatch logs return a max of 10,000 results. This class
* performs a binary search across all of the logs in the provided
* date range if a query returns the maximum number of results.
*
* @param {import('@aws-sdk/client-cloudwatch-logs').CloudWatchLogsClient} client
* @param {{ logGroupNames: string[], dateRange: [Date, Date], queryConfig: { limit: number } }} config
*/
constructor(client, { logGroupNames, dateRange, queryConfig }) {
this.client = client;
/**
* All log groups are queried.
*/
this.logGroupNames = logGroupNames;
/**
* The inclusive date range that is queried.
*/
this.dateRange = dateRange;
/**
* CloudWatch Logs never returns more than 10,000 logs.
*/
this.limit = queryConfig?.limit ?? 10000;
/**
* @type {import("@aws-sdk/client-cloudwatch-logs").ResultField[][]}
*/
this.results = [];
}
/**
* Run the query.
*/
async run() {
this.secondsElapsed = 0;
const start = new Date();
this.results = await this._largeQuery(this.dateRange);
const end = new Date();
this.secondsElapsed = (end - start) / 1000;
return this.results;
}
/**
* Recursively query for logs.
* @param {[Date, Date]} dateRange
* @returns {Promise}
*/
async _largeQuery(dateRange) {
const logs = await this._query(dateRange, this.limit);
console.log(
`Query date range: ${dateRange
.map((d) => d.toISOString())
.join(" to ")}. Found ${logs.length} logs.`,
);
if (logs.length < this.limit) {
return logs;
}
const lastLogDate = this._getLastLogDate(logs);
const offsetLastLogDate = new Date(lastLogDate);
offsetLastLogDate.setMilliseconds(lastLogDate.getMilliseconds() + 1);
const subDateRange = [offsetLastLogDate, dateRange[1]];
const [r1, r2] = splitDateRange(subDateRange);
const results = await Promise.all([
this._largeQuery(r1),
this._largeQuery(r2),
]);
return [logs, ...results].flat();
}
/**
* Find the most recent log in a list of logs.
* @param {import("@aws-sdk/client-cloudwatch-logs").ResultField[][]} logs
*/
_getLastLogDate(logs) {
const timestamps = logs
.map(
(log) =>
log.find((fieldMeta) => fieldMeta.field === "@timestamp")?.value,
)
.filter((t) => !!t)
.map((t) => `${t}Z`)
.sort();
if (!timestamps.length) {
throw new Error("No timestamp found in logs.");
}
return new Date(timestamps[timestamps.length - 1]);
}
/**
* Simple wrapper for the GetQueryResultsCommand.
* @param {string} queryId
*/
_getQueryResults(queryId) {
return this.client.send(new GetQueryResultsCommand({ queryId }));
}
/**
* Starts a query and waits for it to complete.
* @param {[Date, Date]} dateRange
* @param {number} maxLogs
*/
async _query(dateRange, maxLogs) {
try {
const { queryId } = await this._startQuery(dateRange, maxLogs);
const { results } = await this._waitUntilQueryDone(queryId);
return results ?? [];
} catch (err) {
/**
* This error is thrown when StartQuery returns an error indicating
* that the query's start or end date occur before the log group was
* created.
*/
if (err instanceof DateOutOfBoundsError) {
return [];
}
throw err;
}
}
/**
* Wrapper for the StartQueryCommand. Uses a static query string
* for consistency.
* @param {[Date, Date]} dateRange
* @param {number} maxLogs
* @returns {Promise<{ queryId: string }>}
*/
async _startQuery([startDate, endDate], maxLogs = 10000) {
try {
return await this.client.send(
new StartQueryCommand({
logGroupNames: this.logGroupNames,
queryString: "fields @timestamp, @message | sort @timestamp asc",
startTime: startDate.valueOf(),
endTime: endDate.valueOf(),
limit: maxLogs,
}),
);
} catch (err) {
/** @type {string} */
const message = err.message;
if (message.startsWith("Query's end date and time")) {
// This error indicates that the query's start or end date occur
// before the log group was created.
throw new DateOutOfBoundsError(message);
}
throw err;
}
}
/**
* Call GetQueryResultsCommand until the query is done.
* @param {string} queryId
*/
_waitUntilQueryDone(queryId) {
const getResults = async () => {
const results = await this._getQueryResults(queryId);
const queryDone = [
"Complete",
"Failed",
"Cancelled",
"Timeout",
"Unknown",
].includes(results.status);
return { queryDone, results };
};
return retry(
{ intervalInMs: 1000, maxRetries: 60, quiet: true },
async () => {
const { queryDone, results } = await getResults();
if (!queryDone) {
throw new Error("Query not done.");
}
return results;
},
);
}
}
```
+ API 세부 정보는 *AWS SDK for JavaScript API 참조*의 다음 주제를 참조하세요.
+ [GetQueryResults](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/GetQueryResultsCommand)
+ [StartQuery](https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/client/cloudwatch-logs/command/StartQueryCommand)
------
#### [ Python ]
**SDK for Python(Boto3)**
GitHub에 더 많은 내용이 있습니다. [AWS 코드 예 리포지토리](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/cloudwatch-logs/scenarios/large-query#code-examples)에서 전체 예를 찾고 설정 및 실행하는 방법을 배워보세요.
이 파일은 10,000개의 결과를 초과하는 CloudWatch 쿼리를 관리하기 위한 예제 모듈을 간접 호출합니다.
```
import logging
import os
import sys
import boto3
from botocore.config import Config
from cloudwatch_query import CloudWatchQuery
from date_utilities import DateUtilities
# Configure logging at the module level.
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s",
)
DEFAULT_QUERY_LOG_GROUP = "/workflows/cloudwatch-logs/large-query"
class CloudWatchLogsQueryRunner:
def __init__(self):
"""
Initializes the CloudWatchLogsQueryRunner class by setting up date utilities
and creating a CloudWatch Logs client with retry configuration.
"""
self.date_utilities = DateUtilities()
self.cloudwatch_logs_client = self.create_cloudwatch_logs_client()
def create_cloudwatch_logs_client(self):
"""
Creates and returns a CloudWatch Logs client with a specified retry configuration.
:return: A CloudWatch Logs client instance.
:rtype: boto3.client
"""
try:
return boto3.client("logs", config=Config(retries={"max_attempts": 10}))
except Exception as e:
logging.error(f"Failed to create CloudWatch Logs client: {e}")
sys.exit(1)
def fetch_environment_variables(self):
"""
Fetches and validates required environment variables for query start and end dates.
Fetches the environment variable for log group, returning the default value if it
does not exist.
:return: Tuple of query start date and end date as integers and the log group.
:rtype: tuple
:raises SystemExit: If required environment variables are missing or invalid.
"""
try:
query_start_date = int(os.environ["QUERY_START_DATE"])
query_end_date = int(os.environ["QUERY_END_DATE"])
except KeyError:
logging.error(
"Both QUERY_START_DATE and QUERY_END_DATE environment variables are required."
)
sys.exit(1)
except ValueError as e:
logging.error(f"Error parsing date environment variables: {e}")
sys.exit(1)
try:
log_group = os.environ["QUERY_LOG_GROUP"]
except KeyError:
logging.warning("No QUERY_LOG_GROUP environment variable, using default value")
log_group = DEFAULT_QUERY_LOG_GROUP
return query_start_date, query_end_date, log_group
def convert_dates_to_iso8601(self, start_date, end_date):
"""
Converts UNIX timestamp dates to ISO 8601 format using DateUtilities.
:param start_date: The start date in UNIX timestamp.
:type start_date: int
:param end_date: The end date in UNIX timestamp.
:type end_date: int
:return: Start and end dates in ISO 8601 format.
:rtype: tuple
"""
start_date_iso8601 = self.date_utilities.convert_unix_timestamp_to_iso8601(
start_date
)
end_date_iso8601 = self.date_utilities.convert_unix_timestamp_to_iso8601(
end_date
)
return start_date_iso8601, end_date_iso8601
def execute_query(
self,
start_date_iso8601,
end_date_iso8601,
log_group="/workflows/cloudwatch-logs/large-query",
query="fields @timestamp, @message | sort @timestamp asc"
):
"""
Creates a CloudWatchQuery instance and executes the query with provided date range.
:param start_date_iso8601: The start date in ISO 8601 format.
:type start_date_iso8601: str
:param end_date_iso8601: The end date in ISO 8601 format.
:type end_date_iso8601: str
:param log_group: Log group to search: "/workflows/cloudwatch-logs/large-query"
:type log_group: str
:param query: Query string to pass to the CloudWatchQuery instance
:type query: str
"""
cloudwatch_query = CloudWatchQuery(
log_group=log_group,
query_string=query
)
cloudwatch_query.query_logs((start_date_iso8601, end_date_iso8601))
logging.info("Query executed successfully.")
logging.info(
f"Queries completed in {cloudwatch_query.query_duration} seconds. Total logs found: {len(cloudwatch_query.query_results)}"
)
def main():
"""
Main function to start a recursive CloudWatch logs query.
Fetches required environment variables, converts dates, and executes the query.
"""
logging.info("Starting a recursive CloudWatch logs query...")
runner = CloudWatchLogsQueryRunner()
query_start_date, query_end_date, log_group = runner.fetch_environment_variables()
start_date_iso8601 = DateUtilities.convert_unix_timestamp_to_iso8601(
query_start_date
)
end_date_iso8601 = DateUtilities.convert_unix_timestamp_to_iso8601(query_end_date)
runner.execute_query(start_date_iso8601, end_date_iso8601, log_group=log_group)
if __name__ == "__main__":
main()
```
이 모듈은 10,000개의 결과를 초과하는 CloudWatch 쿼리를 처리합니다.
```
import logging
import time
from datetime import datetime
import threading
import boto3
from date_utilities import DateUtilities
DEFAULT_QUERY = "fields @timestamp, @message | sort @timestamp asc"
DEFAULT_LOG_GROUP = "/workflows/cloudwatch-logs/large-query"
class DateOutOfBoundsError(Exception):
"""Exception raised when the date range for a query is out of bounds."""
pass
class CloudWatchQuery:
"""
A class to query AWS CloudWatch logs within a specified date range.
:vartype date_range: tuple
:ivar limit: Maximum number of log entries to return.
:vartype limit: int
:log_group str: Name of the log group to query
:query_string str: query
"""
def __init__(self, log_group: str = DEFAULT_LOG_GROUP, query_string: str=DEFAULT_QUERY) -> None:
self.lock = threading.Lock()
self.log_group = log_group
self.query_string = query_string
self.query_results = []
self.query_duration = None
self.datetime_format = "%Y-%m-%d %H:%M:%S.%f"
self.date_utilities = DateUtilities()
self.limit = 10000
def query_logs(self, date_range):
"""
Executes a CloudWatch logs query for a specified date range and calculates the execution time of the query.
:return: A batch of logs retrieved from the CloudWatch logs query.
:rtype: list
"""
start_time = datetime.now()
start_date, end_date = self.date_utilities.normalize_date_range_format(
date_range, from_format="unix_timestamp", to_format="datetime"
)
logging.info(
f"Original query:"
f"\n START: {start_date}"
f"\n END: {end_date}"
f"\n LOG GROUP: {self.log_group}"
)
self.recursive_query((start_date, end_date))
end_time = datetime.now()
self.query_duration = (end_time - start_time).total_seconds()
def recursive_query(self, date_range):
"""
Processes logs within a given date range, fetching batches of logs recursively if necessary.
:param date_range: The date range to fetch logs for, specified as a tuple (start_timestamp, end_timestamp).
:type date_range: tuple
:return: None if the recursive fetching is continued or stops when the final batch of logs is processed.
Although it doesn't explicitly return the query results, this method accumulates all fetched logs
in the `self.query_results` attribute.
:rtype: None
"""
batch_of_logs = self.perform_query(date_range)
# Add the batch to the accumulated logs
with self.lock:
self.query_results.extend(batch_of_logs)
if len(batch_of_logs) == self.limit:
logging.info(f"Fetched {self.limit}, checking for more...")
most_recent_log = self.find_most_recent_log(batch_of_logs)
most_recent_log_timestamp = next(
item["value"]
for item in most_recent_log
if item["field"] == "@timestamp"
)
new_range = (most_recent_log_timestamp, date_range[1])
midpoint = self.date_utilities.find_middle_time(new_range)
first_half_thread = threading.Thread(
target=self.recursive_query,
args=((most_recent_log_timestamp, midpoint),),
)
second_half_thread = threading.Thread(
target=self.recursive_query, args=((midpoint, date_range[1]),)
)
first_half_thread.start()
second_half_thread.start()
first_half_thread.join()
second_half_thread.join()
def find_most_recent_log(self, logs):
"""
Search a list of log items and return most recent log entry.
:param logs: A list of logs to analyze.
:return: log
:type :return List containing log item details
"""
most_recent_log = None
most_recent_date = "1970-01-01 00:00:00.000"
for log in logs:
for item in log:
if item["field"] == "@timestamp":
logging.debug(f"Compared: {item['value']} to {most_recent_date}")
if (
self.date_utilities.compare_dates(
item["value"], most_recent_date
)
== item["value"]
):
logging.debug(f"New most recent: {item['value']}")
most_recent_date = item["value"]
most_recent_log = log
logging.info(f"Most recent log date of batch: {most_recent_date}")
return most_recent_log
def perform_query(self, date_range):
"""
Performs the actual CloudWatch log query.
:param date_range: A tuple representing the start and end datetime for the query.
:type date_range: tuple
:return: A list containing the query results.
:rtype: list
"""
client = boto3.client("logs")
try:
try:
start_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[0])
)
end_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[1])
)
response = client.start_query(
logGroupName=self.log_group,
startTime=start_time,
endTime=end_time,
queryString=self.query_string,
limit=self.limit,
)
query_id = response["queryId"]
except client.exceptions.ResourceNotFoundException as e:
raise DateOutOfBoundsError(f"Resource not found: {e}")
while True:
time.sleep(1)
results = client.get_query_results(queryId=query_id)
if results["status"] in [
"Complete",
"Failed",
"Cancelled",
"Timeout",
"Unknown",
]:
return results.get("results", [])
except DateOutOfBoundsError:
return []
def _initiate_query(self, client, date_range, max_logs):
"""
Initiates the CloudWatch logs query.
:param date_range: A tuple representing the start and end datetime for the query.
:type date_range: tuple
:param max_logs: The maximum number of logs to retrieve.
:type max_logs: int
:return: The query ID as a string.
:rtype: str
"""
try:
start_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[0])
)
end_time = round(
self.date_utilities.convert_iso8601_to_unix_timestamp(date_range[1])
)
response = client.start_query(
logGroupName=self.log_group,
startTime=start_time,
endTime=end_time,
queryString=self.query_string,
limit=max_logs,
)
return response["queryId"]
except client.exceptions.ResourceNotFoundException as e:
raise DateOutOfBoundsError(f"Resource not found: {e}")
def _wait_for_query_results(self, client, query_id):
"""
Waits for the query to complete and retrieves the results.
:param query_id: The ID of the initiated query.
:type query_id: str
:return: A list containing the results of the query.
:rtype: list
"""
while True:
time.sleep(1)
results = client.get_query_results(queryId=query_id)
if results["status"] in [
"Complete",
"Failed",
"Cancelled",
"Timeout",
"Unknown",
]:
return results.get("results", [])
```
+ API 세부 정보는 *AWS SDK for Python(Boto3) API 참조*의 다음 주제를 참조하세요.
+ [GetQueryResults](https://docs.aws.amazon.com/goto/boto3/logs-2014-03-28/GetQueryResults)
+ [StartQuery](https://docs.aws.amazon.com/goto/boto3/logs-2014-03-28/StartQuery)
------
# 예약된 이벤트를 사용하여 Lambda 함수 간접 호출
다음 코드 예제에서는 Amazon EventBridge 예약 이벤트에서 호출된 AWS Lambda 함수를 생성하는 방법을 보여줍니다.
------
#### [ Java ]
**SDK for Java 2.x**
AWS Lambda 함수를 호출하는 Amazon EventBridge 예약 이벤트를 생성하는 방법을 보여줍니다. Lambda 함수가 간접 호출될 때 cron 표현식을 사용하여 일정을 예약하도록 EventBridge를 구성합니다. 이 예제에서는 Lambda Java 런타임 API를 사용하여 Lambda 함수를 생성합니다. 이 예제에서는 다양한 AWS 서비스를 호출하여 특정 사용 사례를 수행합니다. 이 예제에서는 1주년 기념일에 직원에게 축하하는 모바일 문자 메시지를 전송하는 앱을 생성하는 방법을 보여줍니다.
전체 소스 코드와 설정 및 실행 방법에 대한 지침은 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javav2/usecases/creating_scheduled_events)에서 전체 예제를 참조하세요.
**이 예제에서 사용되는 서비스**
+ CloudWatch Logs
+ DynamoDB
+ EventBridge
+ Lambda
+ Amazon SNS
------
#### [ JavaScript ]
**SDK for JavaScript(v3)**
AWS Lambda 함수를 호출하는 Amazon EventBridge 예약 이벤트를 생성하는 방법을 보여줍니다. Lambda 함수가 간접 호출될 때 cron 표현식을 사용하여 일정을 예약하도록 EventBridge를 구성합니다. 이 예제에서는 Lambda JavaScript 런타임 API를 사용하여 Lambda 함수를 생성합니다. 이 예제에서는 다양한 AWS 서비스를 호출하여 특정 사용 사례를 수행합니다. 이 예제에서는 1주년 기념일에 직원에게 축하하는 모바일 문자 메시지를 전송하는 앱을 생성하는 방법을 보여줍니다.
전체 소스 코드와 설정 및 실행 방법에 대한 지침은 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/javascriptv3/example_code/cross-services/lambda-scheduled-events)에서 전체 예제를 참조하세요.
이 예제는 [AWS SDK for JavaScript v3 개발자 가이드](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/scheduled-events-invoking-lambda-example.html)에서도 확인할 수 있습니다.
**이 예제에서 사용되는 서비스**
+ CloudWatch Logs
+ DynamoDB
+ EventBridge
+ Lambda
+ Amazon SNS
------
#### [ Python ]
**SDK for Python(Boto3)**
이 예제에서는 예약된 Amazon EventBridge 이벤트의 대상으로 AWS Lambda 함수를 등록하는 방법을 보여줍니다. Lambda 핸들러는 나중에 검색할 수 있도록 알기 쉬운 메시지와 전체 이벤트 데이터를 Amazon CloudWatch Logs에 기록합니다.
+ Lambda 함수를 배포합니다.
+ EventBridge 예약된 이벤트를 생성하고 Lambda 함수를 대상으로 만듭니다.
+ EventBridge에 Lambda 함수를 간접 호출할 수 있는 권한을 부여합니다.
+ CloudWatch Logs에서 최신 데이터를 인쇄하여 예약된 간접 호출의 결과를 표시합니다.
+ 데모 중에 생성된 모든 리소스를 정리합니다.
이 예제는 GitHub에서 가장 잘 볼 수 있습니다. 전체 소스 코드와 설정 및 실행 방법에 대한 지침은 [GitHub](https://github.com/awsdocs/aws-doc-sdk-examples/tree/main/python/example_code/lambda#readme)에서 전체 예제를 참조하세요.
**이 예제에서 사용되는 서비스**
+ CloudWatch Logs
+ DynamoDB
+ EventBridge
+ Lambda
+ Amazon SNS
------