# IVS Server-Side Composition \$1 Real-Time Streaming Server-side composition uses an IVS server to mix audio and video from all stage participants and then sends this mixed video to an IVS channel (e.g., to reach a larger audience) or an S3 bucket. Server-side composition is invoked through IVS control-plane operations in the stage’s home region. Broadcasting or recording a stage using server-side composition offers numerous benefits, making it an attractive choice for users seeking efficient and reliable cloud-based video workflows. **Topics** + [Overview of IVS Server-Side Composition](ssc-overview.md) + [Getting Started with IVS Server-Side Composition](ssc-getting-started.md) + [Custom Participant Ordering](ssc-getting-started-custom-participant-ordering.md) + [Enabling Screen Share in IVS Server-Side Composition](ssc-getting-started-screen-share.md) + [Known Issues and Workarounds](ssc-known-issues.md) # Overview of IVS Server-Side Composition This diagram illustrates how server-side composition works: ![\[Broadcasting a stage using server-side composition.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/ssc_Intro_Composite_Recording.png) ## Benefits Compared to client-side composition, server-side composition has the following benefits: + **Reduced client load** — With server-side composition, the burden of processing and combining audio and video sources is shifted from individual client devices to the server itself. Server-side composition eliminates the need for client devices to use their CPU and network resources for compositing the view and transmitting it to IVS. This means viewers can watch the broadcast without their devices having to handle resource-intensive tasks, which can lead to improved battery life and smoother viewing experiences. + **Consistent quality** — Server-side composition allows for precise control over the quality, resolution, and bitrate of the final stream. This ensures a consistent viewing experience for all viewers, regardless of their individual devices' capabilities. + **Resilience** — By centralizing the composition process on the server, the broadcast becomes more robust. Even if a publisher device experiences technical limitations or fluctuations, the server can adapt and provide a smoother stream to all audience members. + **Bandwidth efficiency** — Since the server handles the composition, stage publishers do not have to spend extra bandwidth broadcasting the video to IVS. Alternatively, to broadcast a stage to an IVS channel, you can do the composition client side; see [Enabling Multiple Hosts on an IVS Stream](https://docs.aws.amazon.com//ivs/latest/LowLatencyUserGuide/multiple-hosts.html) in the *IVS Low-Latency Streaming User Guide*. ## Composition Lifecycle Use the diagram below to understand the state transitions of a composition: ![\[Lifecycle of a Server-Side Composition resource.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/ssc_Composition_Lifecycle.png) At a high level, the life cycle of a Composition is as follows: 1. A Composition resource is created when the user calls the StartComposition operation. 1. Once IVS successfully starts the Composition, an “IVS Composition State Change (Session Start)” EventBridge event is sent. See [Using EventBridge with IVS Real-Time Streaming](eventbridge.md) for details about events. 1. Once a Composition is in an active state, the following can happen: + User stops the Composition — If the StopComposition operation is called, IVS initiates a graceful shutdown of the Composition, sending "Destination End" events followed by a "Session End" event. + Composition performs auto-shutdown — If the IVS stage is deleted or no participant is actively publishing to the IVS stage for 60 seconds, the Composition is finalized automatically and EventBridge events are sent. + Destination failure — If a destination unexpectedly fails (e.g., the IVS channel gets deleted), the destination transitions to the `RECONNECTING` state and a “Destination Reconnecting” event is sent. If recovery is impossible, IVS transitions the destination to the `FAILED` state and a “Destination Failure” event is sent. IVS keeps the composition alive if at least one of its destinations is active. 1. Once the composition is in the `STOPPED` or `FAILED` state, it is automatically cleaned up after five minutes. (Then it no longer is retrieved by ListCompositions or GetComposition.) ## IVS API Server-side composition uses these key API elements: + An *EncoderConfiguration* object allows you to customize the format of the video to be generated (height, width, bitrate, and other streaming parameters). You can reuse an EncoderConfiguration every time you call the StartComposition operation. + *Composition* operations track the video composition and output to an IVS channel. + *StorageConfiguration* tracks the S3 bucket where compositions are recorded. To use server-side composition, you need to create an EncoderConfiguration and attach it when calling the StartComposition operation. In this example, the SquareVideo EncoderConfiguration is used in two Compositions: ![\[Server-side composition uses two key API elements.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/ssc_IVS_API_Composite_Recording.png) For complete information, see [IVS Real-Time Streaming API Reference](https://docs.aws.amazon.com//ivs/latest/RealTimeAPIReference/Welcome.html). ## Layouts The StartComposition operation offers two layout options: grid and PiP (Picture-in-Picture). Server-side composition responds to token exchange events in real time. When a participant exchanges a token to update attributes such as featured status or PiP assignment, the composition layout updates automatically without requiring the participant to leave and rejoin the stage. You can also use token exchange to change participant ordering during a live composition. When a participant exchanges their token with an updated order attribute value, the composition re-renders automatically to reflect the new ordering. This eliminates the need for the participant to disconnect and reconnect. For more information, see [Token Exchange](broadcast-mobile-token-exchange.md). ### Grid Layout The grid layout arranges stage participants in a grid of equally sized slots. It provides several customizable properties: + `videoAspectRatio` sets the participant display mode to control the aspect ratio of video tiles. + `videoFillMode` defines how video content fits within the participant tile. + `gridGap` specifies the spacing between participant tiles in pixels. + `omitStoppedVideo` allows excluding stopped video streams from the composition. + `featuredParticipantAttribute` identifies the featured slot. When this is set, the featured participant is displayed in a larger slot on the main screen, with other participants shown below it. + `participantOrderAttribute` enables custom participant ordering based on attribute values in participant tokens. When specified, participants are ordered numerically by their attribute values, with participants lacking the attribute falling back to arrival-time ordering. This provides optional deterministic positioning and allows for role-based layouts. For details on grid layout (including valid values and defaults for all fields), see the [GridConfiguration](https://docs.aws.amazon.com//ivs/latest/RealTimeAPIReference/API_GridConfiguration.html) data type. ![\[Server-side composition grid layout\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/ssc_Grid_Layout.png) ### Picture-in-Picture (PiP) Layout The PiP layout enables displaying a participant in an overlay window with configurable size, position, and behavior. Key properties include: + `pipParticipantAttribute` specifies the participant for the PiP window. + `pipPosition` determines the corner position of the PiP window. + `pipWidth` and `pipHeight`configure the width and height of the PiP window. + `pipOffset` sets the offset position of the PiP window in pixels from the closest edges. + `pipBehavior` defines PiP behavior when all other participants have left. Like the grid layout, the PiP layout supports `featuredParticipantAttribute`, `omitStoppedVideo`, `videoFillMode`, `gridGap`, and `participantOrderAttribute` to further customize the composition. The `participantOrderAttribute` enables custom participant ordering for both selecting the participant for the PiP window and positioning grid participants based on attribute values in participant tokens. For details on PiP layout (including valid values and defaults for all fields), see the [PipConfiguration](https://docs.aws.amazon.com//ivs/latest/RealTimeAPIReference/API_PipConfiguration.html) data type. ![\[Server-side composition Picture-in-Picture (PiP) layout\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/ssc_PiP_Layout.png) **Note**: The maximum resolution supported by a stage publisher on server-side composition is 1080p. If a publisher sends video higher than 1080p, the publisher will be rendered as an audio-only participant. **Important**: Ensure your application does not depend on the specific features of the current layout, such as size and position of tiles. *Visual improvements to layouts can be introduced at any time*. # Getting Started with IVS Server-Side Composition This document takes you through the steps involved in getting started with IVS server-side composition. ## Prerequisites To use server-side composition, you must have a stage with active publishers and use an IVS channel and/or an S3 bucket as the composition destination. To create an S3 bucket, see the S3 documentation on [how to create buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html). The S3 bucket must be created in the same AWS region as the IVS stage. **Important**: If you use an existing S3 bucket: + The **Object Ownership** setting must be **Bucket owner enforced** or **Bucket owner preferred**. + The **Default Encryption** setting must be **Server-side encryption with Amazon S3 managed keys (SSE-S3)**. For details, see the S3 documentation on [controlling ownership of objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html) and [protecting data with encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingEncryption.html). ## API Instructions Below, we describe one possible workflow that uses EventBridge events to start a composition that broadcasts the stage to an IVS channel when a participant publishes. Alternatively, you can start and stop compositions based on your own app logic. See [Composite Recording](rt-composite-recording.md) for another example which showcases the use of server-side composition to record a stage directly to an S3 bucket. 1. Create an IVS channel. See [Getting Started with Amazon IVS Low-Latency Streaming](https://docs.aws.amazon.com//ivs/latest/LowLatencyUserGuide/getting-started.html). 1. Create an IVS stage and participant tokens for each publisher. 1. Create an [EncoderConfiguration](https://docs.aws.amazon.com//ivs/latest/RealTimeAPIReference/API_EncoderConfiguration.html). 1. Join the stage and publish to it. (See the "Publishing and Subscribing" sections of the real-time streaming broadcast SDK guides: [Web](https://docs.aws.amazon.com//ivs/latest/RealTimeUserGuide/web-publish-subscribe.html), [Android](https://docs.aws.amazon.com//ivs/latest/RealTimeUserGuide/android-publish-subscribe.html), and [iOS](https://docs.aws.amazon.com//ivs/latest/RealTimeUserGuide/ios-publish-subscribe.html).) 1. When you receive a Participant Published EventBridge event, call [StartComposition](https://docs.aws.amazon.com//ivs/latest/RealTimeAPIReference/API_StartComposition.html) with your desired layout configuration. 1. Wait for a few seconds and see the composited view in the channel playback. ![\[Server-side composition workflow that uses EventBridge events to start a composition when a participant publishes.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/ssc_Workflow.png) **Note**: A Composition performs auto-shutdown after 60 seconds of inactivity from publisher participants on the stage. At that point, the Composition is terminated and transitions to a `STOPPED` state. A Composition is automatically deleted after a few minutes in the `STOPPED` state. ## CLI Instructions Using the AWS CLI is an advanced option and requires that you first download and configure the CLI on your machine. For details, see the [AWS Command Line Interface User Guide](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html). Now you can use the CLI to create and manage resources. The Composition operations are under the `ivs-realtime` namespace. ### Create the EncoderConfiguration Resource An EncoderConfiguration is an object that allows you to customize the format of the generated video (height, width, bitrate, and other streaming parameters). You can reuse an EncoderConfiguration every time you call the Composition operation, as explained in the next step. The command below creates an EncoderConfiguration resource that configures server-side video composition parameters like video bitrate, frame rate and resolution: ``` aws ivs-realtime create-encoder-configuration --name "MyEncoderConfig" --video "bitrate=2500000,height=720,width=1280,framerate=30" ``` The response is: ``` { "encoderConfiguration": { "arn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/9W59OBY2M8s4", "name": "MyEncoderConfig", "tags": {}, "video": { "bitrate": 2500000, "framerate": 30, "height": 720, "width": 1280 } } } ``` ### Start a Composition Using the EncoderConfiguration ARN provided in the response above, create your Composition resource: **Grid Layout Example** ``` aws ivs-realtime start-composition --stage-arn "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik" --destinations '[{"channel": {"channelArn": "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r", "encoderConfigurationArn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/9W59OBY2M8s4"}}]' --layout '{"grid":{"participantOrderAttribute":"order","featuredParticipantAttribute":"isFeatured","videoFillMode":"COVER","gridGap":0}}' ``` **PiP Layout Example** ``` aws ivs-realtime start-composition --stage-arn "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik" --destinations '[{"channel": {"channelArn": "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r", "encoderConfigurationArn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/DEkQHWPVaOwO"}}]' --layout '{"pip":{"participantOrderAttribute":"priority","pipParticipantAttribute":"isPip","pipOffset":10,"pipPosition":"TOP_RIGHT"}}' ``` **Note**: You can use [this tool](https://composition.ivsdemos.com/) to more easily generate the `--layout` configuration based on your layout choices. The response will show that the Composition is created with a `STARTING` state. Once the Composition starts publishing the composition, the state transitions to `ACTIVE`. (You can see the state by calling the ListCompositions or GetComposition operation.) Once a Composition is `ACTIVE`, the composite view of the IVS stage is visible on the IVS channel, using ListCompositions: ``` aws ivs-realtime list-compositions ``` The response is: ``` { "compositions": [ { "arn": "arn:aws:ivs:us-east-1:927810967299:composition/YVoaXkKdEdRP", "destinations": [ { "id": "bD9rRoN91fHU", "startTime": "2023-09-21T15:38:39+00:00", "state": "ACTIVE" } ], "stageArn": "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik", "startTime": "2023-09-21T15:38:37+00:00", "state": "ACTIVE", "tags": {} } ] } ``` **Note**: You need to have publisher participants actively publishing to the stage to keep the composition alive. For more information, see the "Publishing and Subscribing" sections of the real-time streaming broadcast SDK guides: [Web](https://docs.aws.amazon.com//ivs/latest/RealTimeUserGuide/web-publish-subscribe.html), [Android](https://docs.aws.amazon.com//ivs/latest/RealTimeUserGuide/android-publish-subscribe.html), and [iOS](https://docs.aws.amazon.com//ivs/latest/RealTimeUserGuide/ios-publish-subscribe.html). You must create a distinct stage token for each participant. # Custom Participant Ordering Custom participant ordering allows you to control the positioning of participants in both grid and PiP layouts based on custom attribute values in participant tokens, including the positioning of the featured participants and selection of participants the PiP window. This provides deterministic participant positioning and enables role-based layouts. ## How Custom Ordering Works When `participantOrderAttribute` is specified in your layout configuration, participants are ordered according to the following rules: + Participants with the specified ordering attribute in their tokens are positioned first, sorted numerically by their attribute values. + Participants without the ordering attribute fall back to arrival-time ordering and are positioned after the ordered participants. + When multiple participants have identical ordering values, they are sub-sorted by their arrival time into the stage. + The ordering uses numeric sorting (not lexicographic), so "10" comes after "9" (not after "1"). + Negative values are supported. They are positioned before positive values. + Non-numeric values (e.g., "abc", "1.5") are treated as invalid, and those participants fall back to arrival-time ordering. **Important:** Participant ordering (whether based on the arrival time or custom) takes effect after the composition has started. The correct ordering of participants is not guaranteed for participants who join the stage before the composition begins. ## Creating Tokens with Ordering Attributes To use custom participant ordering, include the ordering attribute in your participant tokens when creating them: ``` aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes order=1 aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes order=2 aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes order=3 ``` You can combine the custom-participant-order attribute with the attributes for selecting participants for the featured slot and the PiP window: ``` aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes order=2,isFeatured=true aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes order=3,isFeatured=true aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes order=4,isPip=true ``` ## Example Use Cases Example use cases include: + Consistent positioning – Participants maintain their positions when reconnecting with the same token. + Role-based positioning – For instance, you could specify teachers with order=1 and students with order=2. + Priority-based layouts – VIP participants with lower order values would appear first. + Dynamic layouts – You can combine custom ordering with `featuredParticipantAttribute` and `pipParticipantAttribute` for complex scenarios. + Cross-stage interactions – When using participant replication for scenarios like VS Mode competitions where streamers from different stages interact, you can override ordering attributes to control positioning in the destination stage composition. **Note:** For participant-replication use cases, you can override participant attributes (including the order attribute) as needed when starting a replication to achieve the desired layout in the destination stage. ## Backward Compatibility Custom participant ordering is an optional feature and is fully backward compatible. Existing compositions without `participantOrderAttribute` continue to work unchanged, using arrival-time ordering. When `participantOrderAttribute` is set to an empty string, the system ignores custom ordering entirely and falls back to default behavior. # Enabling Screen Share in IVS Server-Side Composition To use a fixed screen-share layout, follow the steps below. ## Create the EncoderConfiguration Resource The command below creates an EncoderConfiguration resource that configures server-side composition parameters (video bitrate, framerate, and resolution). ``` aws ivs-realtime create-encoder-configuration --name "test-ssc-with-screen-share" --video={bitrate=2000000,framerate=30,height=720,width=1280} ``` Create a stage participant token with a `screen-share` attribute. Since we will specify `screen-share` as the name of the `featured` slot, we need to create a stage token with the `screen-share` attribute set to `true`: ``` aws ivs-realtime create-participant-token --stage-arn "arn:aws:ivs:us-east-1:123456789012:stage/u9OiE29bT7Xp" --attributes screen-share=true ``` The response is: ``` { "participantToken": { "attributes": { "screen-share": "true" }, "expirationTime": "2023-08-04T05:26:11+00:00", "participantId": "E813MFklPWLF", "token": "eyJhbGciOiJLTVMiLCJ0eXAiOiJKV1QifQ.eyJleHAiOjE2OTExMjY3NzEsImlhdCI6MTY5MTA4MzU3MSwianRpIjoiRTgxM01Ga2xQV0xGIiwicmVzb3VyY2UiOiJhcm46YXdzOml2czp1cy1lYXN0LTE6OTI3ODEwOTY3Mjk5OnN0YWdlL3U5T2lFMjliVDdYcCIsInRvcGljIjoidTlPaUUyOWJUN1hwIiwiZXZlbnRzX3VybCI6IndzczovL3VzLWVhc3QtMS5ldmVudHMubGl2ZS12aWRlby5uZXQiLCJ3aGlwX3VybCI6Imh0dHBzOi8vYjJlYTVjMmZmMzU1Lmdsb2JhbC53aGlwLmxpdmUtdmlkZW8ubmV0IiwiYXR0cmlidXRlcyI6eyJzY3JlZW4tc2hhcmUiOiJ0cnVlIn0sImNhcGFiaWxpdGllcyI6eyJhbGxvd19wdWJsaXNoIjp0cnVlLCJhbGxvd19zdWJzY3JpYmUiOnRydWV9LCJ2ZXJzaW9uIjoiMC4zIn0.MGUCMFvMzv35O4yVzM9tIWZl7n3mmFQhleqsRSBx_G2qT2YUDlWSNg6H1vL7sAWQMeydSAIxAIvdfqt3Fh1MLiyelc9NnTjI5hL3YPKqDX6J3NDH1fksh8_5y1jztoPDy4yVA5OmtA" } } ``` ## Start the Composition To start the composition using the screen-share feature, we use this command: ``` aws ivs-realtime start-composition --stage-arn "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik" --destinations '[{"channel": {"channelArn": "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r", "encoderConfigurationArn": "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/DEkQHWPVaOwO"}}]' --layout '{"grid":{"featuredParticipantAttribute":"screen-share"}}' ``` The response is: ``` { "composition" : { "arn" : "arn:aws:ivs:us-east-1:927810967299:composition/B19tQcXRgtoz", "destinations" : [ { "configuration" : { "channel" : { "channelArn" : "arn:aws:ivs:us-east-1:927810967299:channel/DOlMW4dfMR8r", "encoderConfigurationArn" : "arn:aws:ivs:us-east-1:927810967299:encoder-configuration/DEkQHWPVaOwO" }, "name" : "" }, "id" : "SGmgBXTULuXv", "state" : "STARTING" } ], "layout" : { "grid" : { "featuredParticipantAttribute" : "screen-share", "gridGap": 2, "omitStoppedVideo": false, "videoAspectRatio": "VIDEO" } }, "stageArn" : "arn:aws:ivs:us-east-1:927810967299:stage/8faHz1SQp0ik", "startTime" : "2023-09-27T21:32:38Z", "state" : "STARTING", "tags" : { } } } ``` When the stage participant `E813MFklPWLF` joins the stage, that participant’s video will be displayed in the featured slot, and all other stage publishers will be rendered below the slot: ![\[Starting the Composition using the screen-share feature.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/ssc_StartComposition.png) ## Stop the Composition To stop a composition at any point, call the StopComposition operation: ``` aws ivs-realtime stop-composition --arn arn:aws:ivs:us-east-1:927810967299:composition/B19tQcXRgtoz ``` # Known Issues and Workarounds This section lists known issues that you might encounter when using IVS server-side composition and suggests potential workarounds. + Some compositions may exhibit brief audio stuttering after periods of silence. **Workaround:** None