Extract images from video streams

You can use Amazon Kinesis Video Streams APIs and SDKs to help you extract images from your video streams. You can use these images for enhanced playback applications such as thumbnails or enhanced scrubbing, or for use in machine learning pipelines. Kinesis Video Streams offers on-demand image extraction through an API, or automated image extraction from metadata tags in an ingested video.

For information about using Kinesis Video Streams managed support for images, see:

Automated image generation (Amazon S3 delivery)

Currently, customers run and manage their own image transcoding pipeline to create images for various purposes like scrubbing, image preview, running ML models on images, and more. Kinesis Video Streams offers the capability to transcode and deliver the images. Kinesis Video Streams will automatically extract images from video data in real-time based on a tag, and deliver the images to a customer specified S3 bucket.

UpdateImageGenerationConfiguration

{
 "StreamName": "TestStream",
 "ImageGenerationConfiguration":
 {
  "Status": "ENABLED",
  "DestinationConfig":
  {
   "DestinationRegion": "us-east-1",
   "Uri": "s3://bucket-name"
  },
  "SamplingInterval": 200,
  "ImageSelectorType": "PRODUCER_TIMESTAMP",
  "Format": "JPEG",
  "FormatConfig": {
                "JPEGQuality": "80"
       },
  "WidthPixels": 320,
  "HeightPixels": 240
 }
}

You can use the AWS CLI to invoke the UpdateImageGenerationConfiguration API operation to add the Amazon S3 ARN created previously and change the status to ENABLED.

aws kinesisvideo update-image-generation-configuration \
--cli-input-json file://./update-image-generation-input.json \

Request:

UpdateImageGenerationConfiguration HTTP/1.1
                 
  Method: 'POST'
  Path: '/updateImageGenerationConfiguration' 
  Body: {
    StreamName: 'String', // Optional. Either stream name or arn should be passed
    StreamArn: 'String', // Optional. Either stream name or arn should be passed
    ImageGenerationConfiguration : {
        // required
        Status: 'Enum', // ENABLED | DISABLED,
       ImageSelectorType: 'Enum', // SERVER_TIMESTAMP | PRODUCER_TIMESTAMP..      
        DestinationConfig: {
            DestinationRegion: 'String', 
            Uri: string,
        },
        SamplingInterval: 'Number'//
        Format: 'Enum', // JPEG | PNG
        // Optional parameters
        FormatConfig: {
                'String': 'String',
       },
        WidthPixels: 'Number', // 1 - 3840 (4k).   
        HeightPixels: 'Number' // 1 - 2160 (4k).  
    }
  }
  

Response:

  HTTP/1.1 200 
  Content-type: application/json
  Body: {
  }  
            

DescribeImageGenerationConfiguration

To view image generation configurations that are already set for a stream, customers can make a DescribeImageGenerationConfiguration request, as follows.

Request:

 DescribeImageGenerationConfiguration HTTP/1.1

 
  Method: 'POST'
  Path: '/describeImageGenerationConfiguration' 
  Body: {
    StreamName: 'String', // Optional. Either stream name or arn should be passed
    StreamArn: 'String', // Optional. Either stream name or arn should be passed
  }
        

Response:

HTTP/1.1 200 
  Content-type: application/json
  Body: {
    ImageGenerationConfiguration : {
        Status: 'Enum',
        ImageSelectorType: 'Enum', // SERVER_TIMESTAMP | PRODUCER_TIMESTAMP
        DestinationConfig: {
            DestinationRegion: 'String'
            Uri: 'string',
        },
        SamplingInterval: 'Number',
        Format: 'Enum', 
        FormatConfig: {
                'String': 'String',
        },
        WidthPixels: 'Number',
        HeightPixels: 'Number'
    }
  } 

To learn more about the DescribeImageGenerationConfiguration feature, see DescribeImageGenerationConfiguration in the Amazon Kinesis Video Streams Developer Guide.

Producer MKV tags

You can use the Kinesis Video Streams producer SDK to tag specific fragments of interest by exposing an API operation in the SDK. For an example of a tag, see this code. Upon calling this API, the SDK will add a set of predefined MKV tags along with the fragment data. Kinesis Video Streams will recognize these special MKV tags and initiate the image generation workflow based on the image processing configuration of that stream.

Any fragment metadata provided along with the Amazon S3 image generation tags will be saved as Amazon S3 metadata.

Syntax for producer MKV tags

|+ Tags
| + Tag
|  // MANDATORY: Predefined MKV tag to trigger image generation for the fragment
|  + Simple
|   + Name: AWS_KINESISVIDEO_IMAGE_GENERATION

|  // OPTIONAL: S3 prefix which will be set as prefix for generated image.
|  + Simple
|   + Name: AWS_KINESISVIDEO_IMAGE_PREFIX 
|   + String: image_prefix_in_s3 // 256 bytes max m

| // OPTIONAL: Key value pairs that will be persisted as S3 Image object metadata.
|  + Simple
|   + Name: CUSTOM_KEY_1 // Max 128 bytes
|   + String:CUSTOM_VALUE_1 // Max 256 bytes
|  + Simple
|   + Name: CUSTOM_KEY_2 // Max 128 bytes
|   + String: CUSTOM_VALUE_2 // Max 256 bytes
           

Add metadata tags in producer SDK using PutEventMetaData

The PutEventMetaData function appends an MKV file that's associated with an event. PutEventMetaData takes two parameters. The first parameter is an event whose value comes from the STREAM_EVENT_TYPE enum. The second parameter, pStreamEventMetadata, is optional and can be used to include additional metadata as key-value pairs. There is a limit of five key-value pairs of metadata that can be added.

Limits

The following table lists the limitations associated with the metadata tags. If the metadata tag limit is adjustable, you can request an increase through your account manager.

S3 Object metadata

By default, Kinesis Video Streams will set the fragment number, producer, and server timestamp of the image generated as Amazon S3 object metadata. If any additional fragment data is specified in the MKV tags, those tags will also be added to the Amazon S3 object metadata. The following example shows the correct syntax for the Amazon S3 object metadata.

 {
  // KVS S3 object metadata
  x-amz-meta-aws_kinesisvideo_fragment_number : 'string', 
  x-amz-meta-aws_kinesisvideo_producer_timestamp: 'number',
  x-amz-meta-aws_kinesisvideo_server_timestamp: 'number',
  
  // Optional key value pair sent as part of the MKV tags
  custom_key_1: custom_value_1,
  custom_key_2: custom_value_2,
}
                

Amazon S3 object path (image)

The following list shows the correct format of the object path and describes each element within the path.

Format:

ImagePrefix_AccountID_StreamName_ImageTimecode_RandomID.file-extension

1.ImagePrefix - Value of AWS_KINESISVIDEO_IMAGE_PREFIX.

2.AccountID - Account ID under which the stream is created.

3.StreamName - Name of the stream for which the image is generated.

4.ImageTimecode - Epoch timecode in the fragment at which the image is generated.

5.RandomID - Random GUID.

6.file-extension - JPG or PNG based on the image format requested.

Amazon S3 URI recommendations to protect against throttling

If you write thousands of images to Amazon S3, there's a risk of throttling. For more information, see S3 Prefix Put Request Limits.

An Amazon S3 prefix starts with a PUT limit of 3,500 PUT requests per second, and will gradually ramp up over time for unique prefixes. Avoid using dates and times as Amazon S3 prefixes. Time coded data will impact one prefix at a time, and will also change regularly, invalidating previous prefix scale ups. To enable faster, consistent Amazon S3 scaling, we recommend adding a random prefix, like a hex code or UUID to the Amazon S3 Destination URI. For example, hex code prefixes will naturally split your requests randomly among 16 different prefixes (a prefix for each unique hex character), which will allow a 56,000 PUT requests per second after Amazon S3 has auto-scaled.