Creating delivery streams using the Firehose API and the AWS SDK for PHP Version 3

Amazon Data Firehose enables you to send real-time data to other AWS services including Amazon Kinesis Data Streams, Amazon S3, Amazon OpenSearch Service (OpenSearch Service), and Amazon Redshift, or to Splunk. Create a data producer with delivery streams to deliver data to the configured destination every time you add data.

The following examples show how to:

All the example code for the AWS SDK for PHP is available here on GitHub.

Credentials

Before running the example code, configure your AWS credentials, as described in Credentials. Then import the AWS SDK for PHP, as described in Basic usage.

For more information about using Amazon Data Firehose, see the Amazon Kinesis Data Firehose Developer Guide.

Create a delivery stream using a Kinesis data stream

To establish a delivery stream that puts data into an existing Kinesis data stream, use the CreateDeliveryStream operation.

This enables developers to migrate existing Kinesis services to Firehose.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

$name = "my_stream_name";
$stream_type = "KinesisStreamAsSource";
$kinesis_stream = "arn:aws:kinesis:us-east-2:0123456789:stream/my_stream_name";
$role = "arn:aws:iam::0123456789:policy/Role";

try {
    $result = $firehoseClient->createDeliveryStream([
        'DeliveryStreamName' => $name,
        'DeliveryStreamType' => $stream_type,
        'KinesisStreamSourceConfiguration' => [
            'KinesisStreamARN' => $kinesis_stream,
            'RoleARN' => $role,
        ],
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}

Create a delivery stream using an Amazon S3 bucket

To establish a delivery stream that puts data into an existing Amazon S3 bucket, use the CreateDeliveryStream operation.

Provide the destination parameters, as described in Destination Parameters. Then ensure that you grant Firehose access to your Amazon S3 bucket, as described in Grant Kinesis Data Firehose Access to an Amazon S3 Destination.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

$name = "my_S3_stream_name";
$stream_type = "DirectPut";
$s3bucket = 'arn:aws:s3:::bucket_name';
$s3Role = 'arn:aws:iam::0123456789:policy/Role';

try {
    $result = $firehoseClient->createDeliveryStream([
        'DeliveryStreamName' => $name,
        'DeliveryStreamType' => $stream_type,
        'S3DestinationConfiguration' => [
            'BucketARN' => $s3bucket,
            'CloudWatchLoggingOptions' => [
                'Enabled' => false,
            ],
            'RoleARN' => $s3Role
        ],
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}

Create a delivery stream using OpenSearch Service

To establish a Firehose delivery stream that puts data into an OpenSearch Service cluster, use the CreateDeliveryStream operation.

Provide the destination parameters, as described in Destination Parameters. Ensure that you grant Firehose access to your OpenSearch Service cluster, as described in Grant Kinesis Data Firehose Access to an Amazon ES Destination.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

$name = "my_ES_stream_name";
$stream_type = "DirectPut";
$esDomainARN = 'arn:aws:es:us-east-2:0123456789:domain/Name';
$esRole = 'arn:aws:iam::0123456789:policy/Role';
$esIndex = 'root';
$esType = 'PHP_SDK';
$s3bucket = 'arn:aws:s3:::bucket_name';
$s3Role = 'arn:aws:iam::0123456789:policy/Role';

try {
    $result = $firehoseClient->createDeliveryStream([
        'DeliveryStreamName' => $name,
        'DeliveryStreamType' => $stream_type,
        'ElasticsearchDestinationConfiguration' => [
            'DomainARN' => $esDomainARN,
            'IndexName' => $esIndex,
            'RoleARN' => $esRole,
            'S3Configuration' => [
                'BucketARN' => $s3bucket,
                'CloudWatchLoggingOptions' => [
                    'Enabled' => false,
                ],
                'RoleARN' => $s3Role,
            ],
            'TypeName' => $esType,
        ],
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}

Retrieve a delivery stream

To get the details about an existing Firehose delivery stream, use the DescribeDeliveryStream operation.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

$name = "my_stream_name";

try {
    $result = $firehoseClient->describeDeliveryStream([
        'DeliveryStreamName' => $name,
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}

List existing delivery streams connected to Kinesis Data Streams

To list all the existing Firehose delivery streams sending data to Kinesis Data Streams, use the ListDeliveryStreams operation.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

try {
    $result = $firehoseClient->listDeliveryStreams([
        'DeliveryStreamType' => 'KinesisStreamAsSource',
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}

List existing delivery streams sending data to other AWS services

To list all the existing Firehose delivery streams sending data to Amazon S3, OpenSearch Service, or Amazon Redshift, or to Splunk, use the ListDeliveryStreams operation.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

try {
    $result = $firehoseClient->listDeliveryStreams([
        'DeliveryStreamType' => 'DirectPut',
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}

Send data to an existing Firehose delivery stream

To send data through a Firehose delivery stream to your specified destination, use the PutRecord operation after you create a Firehose delivery stream.

Before sending data to a Firehose delivery stream, use DescribeDeliveryStream to see if the delivery stream is active.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

$name = "my_stream_name";
$content = '{"ticker_symbol":"QXZ", "sector":"HEALTHCARE", "change":-0.05, "price":84.51}';

try {
    $result = $firehoseClient->putRecord([
        'DeliveryStreamName' => $name,
        'Record' => [
            'Data' => $content,
        ],
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}

Delete a Firehose delivery stream

To delete a Firehose delivery stream, use the DeleteDeliveryStreams operation. This also deletes any data you have sent to the delivery stream.

Imports

require 'vendor/autoload.php';

use Aws\Exception\AwsException;

Sample Code

$firehoseClient = new Aws\Firehose\FirehoseClient([
    'profile' => 'default',
    'version' => '2015-08-04',
    'region' => 'us-east-2'
]);

$name = "my_stream_name";

try {
    $result = $firehoseClient->deleteDeliveryStream([
        'DeliveryStreamName' => $name,
    ]);
    var_dump($result);
} catch (AwsException $e) {
    // output error message if fails
    echo $e->getMessage();
    echo "\n";
}