本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。
搭配 Fluent Bit OpenSearch 使用擷取管道
此範例 Fluent Bit 組態檔案
注意下列事項:
-
host
值必須是您的管道端點。例如:
。pipeline-endpoint
.us-east-1.osis.amazonaws.com -
aws_service
值必須為osis
。 -
此
aws_role_arn
值是用戶端擔任和用於 Signature 第 AWS IAM4 版身分驗證ARN的角色。
[INPUT] name tail refresh_interval 5 path /var/log/test.log read_from_head true [OUTPUT] Name http Match * Host
pipeline-endpoint
.us-east-1
.osis.amazonaws.com Port 443 URI /log/ingest Format json aws_auth true aws_regionus-east-1
aws_service osis aws_role_arn arn:aws:iam::{account-id}
:role/ingestion-role
Log_Level trace tls On
然後,您可以設定 OpenSearch 如下所示的擷取管道,其中具有 HTTP作為來源:
version: "2" unaggregated-log-pipeline: source: http: path: "/log/ingest" processor: - grok: match: log: - "%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE:network_node} %{NOTSPACE:network_host} %{IPORHOST:source_ip}:%{NUMBER:source_port:int} -> %{IPORHOST:destination_ip}:%{NUMBER:destination_port:int} %{GREEDYDATA:details}" - grok: match: details: - "'%{NOTSPACE:http_method} %{NOTSPACE:http_uri}' %{NOTSPACE:protocol}" - "TLS%{NOTSPACE:tls_version} %{GREEDYDATA:encryption}" - "%{NUMBER:status_code:int} %{NUMBER:response_size:int}" - delete_entries: with_keys: ["details", "log"] sink: - opensearch: hosts: ["https://search-
domain-endpoint
.us-east-1
.es.amazonaws.com"] index: "index_name
" index_type: custom bulk_size: 20 aws: # IAM role that the pipeline assumes to access the domain sink sts_role_arn: "arn:aws:iam::{account-id}
:role/pipeline-role
" region: "us-east-1
"