# Step 5: Publish and Subscribe to Video
You can publish/subscribe (real-time) to IVS with:
+ The native [IVS broadcast SDKs](https://docs.aws.amazon.com//ivs/latest/LowLatencyUserGuide/getting-started-set-up-streaming.html#broadcast-sdk), which support WebRTC and RTMPS. We recommend this, especially for production scenarios. See the details below for [Web](getting-started-pub-sub-web.md), [Android](getting-started-pub-sub-android.md), and [iOS](getting-started-pub-sub-ios.md).
+ The Amazon IVS console — This is suitable for testing streams. See below.
+ Other streaming software and hardware encoders — You can use any streaming encoder that supports the RTMP, RTMPS, or WHIP protocols. See [Stream Ingest](rt-stream-ingest.md) for more information.
## IVS Console
1. Open the [Amazon IVS console](https://console.aws.amazon.com/ivs).
(You can also access the Amazon IVS console through the [AWS Management Console](https://console.aws.amazon.com/).)
1. In the navigation pane, select **Stages**. (If the nav pane is collapsed, expand it by selecting the hamburger icon.)
1. Select the stage to which you want to subscribe or publish, to go to its details page.
1. To subscribe: If the stage has one or more publishers, you can subscribe to it by pressing the **Subscribe** button, under the **Subscribe** tab. (Tabs are below the **General Configuration** section.)
1. To publish:
1. Select the **Publish** tab.
1. You will be prompted to grant the IVS console access to your camera and microphone; **Allow** those permissions.
1. Toward the bottom of the **Publish** tab, use the dropdown boxes to select input devices for the microphone and camera.
1. To begin publishing, select **Start publishing**.
1. To view your published content, go back to the **Subscribe** tab.
1. To stop publishing, go to the **Publish** tab and press the **Stop publishing** button towards the bottom.
**Note**: Subscribing and publishing consume resources, and you will incur an hourly rate for the time you are connected to the stage. To learn more, see [Real-Time Streaming](https://aws.amazon.com/ivs/pricing/#Real-Time_Streaming) on the IVS Pricing page.
# Publish & Subscribe with the IVS Web Broadcast SDK
This section takes you through the steps involved in publishing and subscribing to a stage using your web app.
## Create HTML Boilerplate
First let's create the HTML boilerplate and import the library as a script tag:
```
```
## Accept Token Input and Add Join/Leave Buttons
Here we fill in the body with our input controls. These take as input the token, and they set up **Join** and **Leave** buttons. Typically applications will request the token from your application's API, but for this example you'll copy and paste the token into the token input.
```
IVS Real-Time Streaming
```
## Add Media Container Elements
These elements will hold the media for our local and remote participants. We add a script tag to load our application's logic defined in `app.js`.
```
```
This completes the HTML page and you should see this when loading `index.html` in a browser:
![\[View Real-Time Streaming in a browser: HTML setup complete.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/RT_Browser_View.png)
## Create app.js
Let's move to defining the contents of our `app.js` file. Begin by importing all the requisite properties from the SDK's global:
```
const {
Stage,
LocalStageStream,
SubscribeType,
StageEvents,
ConnectionState,
StreamType
} = IVSBroadcastClient;
```
## Create Application Variables
Establish variables to hold references to our **Join** and **Leave** button HTML elements and store state for the application:
```
let joinButton = document.getElementById("join-button");
let leaveButton = document.getElementById("leave-button");
// Stage management
let stage;
let joining = false;
let connected = false;
let localCamera;
let localMic;
let cameraStageStream;
let micStageStream;
```
## Create joinStage 1: Define the Function and Validate Input
The `joinStage` function takes the input token, creates a connection to the stage, and begins to publish video and audio retrieved from `getUserMedia`.
To start, we define the function and validate the state and token input. We'll flesh out this function in the next few sections.
```
const joinStage = async () => {
if (connected || joining) {
return;
}
joining = true;
const token = document.getElementById("token").value;
if (!token) {
window.alert("Please enter a participant token");
joining = false;
return;
}
// Fill in with the next sections
};
```
## Create joinStage 2: Get Media to Publish
Here is the media that will be published to the stage:
```
async function getCamera() {
// Use Max Width and Height
return navigator.mediaDevices.getUserMedia({
video: true,
audio: false
});
}
async function getMic() {
return navigator.mediaDevices.getUserMedia({
video: false,
audio: true
});
}
// Retrieve the User Media currently set on the page
localCamera = await getCamera();
localMic = await getMic();
// Create StageStreams for Audio and Video
cameraStageStream = new LocalStageStream(localCamera.getVideoTracks()[0]);
micStageStream = new LocalStageStream(localMic.getAudioTracks()[0]);
```
## Create joinStage 3: Define the Stage Strategy and Create the Stage
This stage strategy is the heart of the decision logic that the SDK uses to decide what to publish and which participants to subscribe to. For more information on the function's purpose, see [Strategy](web-publish-subscribe.md#web-publish-subscribe-concepts-strategy).
This strategy is simple. After joining the stage, publish the streams we just retrieved and subscribe to every remote participant's audio and video:
```
const strategy = {
stageStreamsToPublish() {
return [cameraStageStream, micStageStream];
},
shouldPublishParticipant() {
return true;
},
shouldSubscribeToParticipant() {
return SubscribeType.AUDIO_VIDEO;
}
};
stage = new Stage(token, strategy);
```
## Create joinStage 4: Handle Stage Events and Render Media
Stages emit many events. We'll need to listen to the `STAGE_PARTICIPANT_STREAMS_ADDED` and `STAGE_PARTICIPANT_LEFT` to render and remove media to and from the page. A more exhaustive set of events are listed in [Events](web-publish-subscribe.md#web-publish-subscribe-concepts-events).
Note that we create four helper functions here to assist us in managing necessary DOM elements: `setupParticipant`, `teardownParticipant`, `createVideoEl`, and `createContainer`.
```
stage.on(StageEvents.STAGE_CONNECTION_STATE_CHANGED, (state) => {
connected = state === ConnectionState.CONNECTED;
if (connected) {
joining = false;
joinButton.style = "display: none";
leaveButton.style = "display: inline-block";
}
});
stage.on(
StageEvents.STAGE_PARTICIPANT_STREAMS_ADDED,
(participant, streams) => {
console.log("Participant Media Added: ", participant, streams);
let streamsToDisplay = streams;
if (participant.isLocal) {
// Ensure to exclude local audio streams, otherwise echo will occur
streamsToDisplay = streams.filter(
(stream) => stream.streamType === StreamType.VIDEO
);
}
const videoEl = setupParticipant(participant);
streamsToDisplay.forEach((stream) =>
videoEl.srcObject.addTrack(stream.mediaStreamTrack)
);
}
);
stage.on(StageEvents.STAGE_PARTICIPANT_LEFT, (participant) => {
console.log("Participant Left: ", participant);
teardownParticipant(participant);
});
// Helper functions for managing DOM
function setupParticipant({ isLocal, id }) {
const groupId = isLocal ? "local-media" : "remote-media";
const groupContainer = document.getElementById(groupId);
const participantContainerId = isLocal ? "local" : id;
const participantContainer = createContainer(participantContainerId);
const videoEl = createVideoEl(participantContainerId);
participantContainer.appendChild(videoEl);
groupContainer.appendChild(participantContainer);
return videoEl;
}
function teardownParticipant({ isLocal, id }) {
const groupId = isLocal ? "local-media" : "remote-media";
const groupContainer = document.getElementById(groupId);
const participantContainerId = isLocal ? "local" : id;
const participantDiv = document.getElementById(
participantContainerId + "-container"
);
if (!participantDiv) {
return;
}
groupContainer.removeChild(participantDiv);
}
function createVideoEl(id) {
const videoEl = document.createElement("video");
videoEl.id = id;
videoEl.autoplay = true;
videoEl.playsInline = true;
videoEl.srcObject = new MediaStream();
return videoEl;
}
function createContainer(id) {
const participantContainer = document.createElement("div");
participantContainer.classList = "participant-container";
participantContainer.id = id + "-container";
return participantContainer;
}
```
## Create joinStage 5: Join the Stage
Let's complete our `joinStage` function by finally joining the stage\$1
```
try {
await stage.join();
} catch (err) {
joining = false;
connected = false;
console.error(err.message);
}
```
## Create leaveStage
Define the `leaveStage` function which the leave button will invoke.
```
const leaveStage = async () => {
stage.leave();
joining = false;
connected = false;
};
```
## Initialize Input-Event Handlers
We'll add one last function to our `app.js` file. This function is invoked immediately when the page loads and establishes event handlers for joining and leaving the stage.
```
const init = async () => {
try {
// Prevents issues on Safari/FF so devices are not blank
await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
} catch (e) {
alert(
"Problem retrieving media! Enable camera and microphone permissions."
);
}
joinButton.addEventListener("click", () => {
joinStage();
});
leaveButton.addEventListener("click", () => {
leaveStage();
joinButton.style = "display: inline-block";
leaveButton.style = "display: none";
});
};
init(); // call the function
```
## Run the Application and Provide a Token
At this point you can share the web page locally or with others, [open the page](#getting-started-pub-sub-web-media), and put in a participant token and join the stage.
## What’s Next?
For more detailed examples involving npm, React, and more, see the [IVS Broadcast SDK: Web Guide (Real-Time Streaming Guide)](broadcast-web.md).
# Publish & Subscribe with the IVS Android Broadcast SDK
This section takes you through the steps involved in publishing and subscribing to a stage using your Android app.
## Create Views
We start by creating a simple layout for our app using the auto-created `activity_main.xml` file. The layout contains an `EditText` to add a token, a Join `Button`, a `TextView` to show the stage state, and a `CheckBox` to toggle publishing.
![\[Set up the publishing layout for your Android app.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_Android_1.png)
Here is the XML behind the view:
```
```
We referenced a couple of string IDs here, so we’ll create our entire `strings.xml` file now:
```
BasicRealTimeJoinLeaveParticipant TokenPublishState: %1$s
```
Let’s link those views in the XML to our `MainActivity.kt`:
```
import android.widget.Button
import android.widget.CheckBox
import android.widget.EditText
import android.widget.TextView
import androidx.recyclerview.widget.RecyclerView
private lateinit var checkboxPublish: CheckBox
private lateinit var recyclerView: RecyclerView
private lateinit var buttonJoin: Button
private lateinit var textViewState: TextView
private lateinit var editTextToken: EditText
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
setContentView(R.layout.activity_main)
checkboxPublish = findViewById(R.id.main_publish_checkbox)
recyclerView = findViewById(R.id.main_recycler_view)
buttonJoin = findViewById(R.id.main_join)
textViewState = findViewById(R.id.main_state)
editTextToken = findViewById(R.id.main_token)
}
```
Now we create an item view for our `RecyclerView`. To do this, right-click your `res/layout` directory and select **New > Layout Resource File**. Name this new file `item_stage_participant.xml`.
![\[Create an item view for your Android app RecyclerView.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_Android_2.png)
The layout for this item is simple: it contains a view for rendering a participant’s video stream and a list of labels for displaying information about the participant:
![\[Create an item view for your Android app RecyclerView - labels.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_Android_3.png)
Here is the XML:
```
```
This XML file inflates a class we haven’t created yet, `ParticipantItem`. Because the XML includes the full namespace, be sure to update this XML file to your namespace. Let’s create this class and set up the views, but otherwise leave it blank for now.
Create a new Kotlin class, `ParticipantItem`:
```
package com.amazonaws.ivs.realtime.basicrealtime
import android.content.Context
import android.util.AttributeSet
import android.widget.FrameLayout
import android.widget.TextView
import kotlin.math.roundToInt
class ParticipantItem @JvmOverloads constructor(
context: Context,
attrs: AttributeSet? = null,
defStyleAttr: Int = 0,
defStyleRes: Int = 0,
) : FrameLayout(context, attrs, defStyleAttr, defStyleRes) {
private lateinit var previewContainer: FrameLayout
private lateinit var textViewParticipantId: TextView
private lateinit var textViewPublish: TextView
private lateinit var textViewSubscribe: TextView
private lateinit var textViewVideoMuted: TextView
private lateinit var textViewAudioMuted: TextView
private lateinit var textViewAudioLevel: TextView
override fun onFinishInflate() {
super.onFinishInflate()
previewContainer = findViewById(R.id.participant_preview_container)
textViewParticipantId = findViewById(R.id.participant_participant_id)
textViewPublish = findViewById(R.id.participant_publishing)
textViewSubscribe = findViewById(R.id.participant_subscribed)
textViewVideoMuted = findViewById(R.id.participant_video_muted)
textViewAudioMuted = findViewById(R.id.participant_audio_muted)
textViewAudioLevel = findViewById(R.id.participant_audio_level)
}
}
```
## Permissions
To use the camera and microphone, you need to request permissions from the user. We follow a standard permissions flow for this:
```
override fun onStart() {
super.onStart()
requestPermission()
}
private val requestPermissionLauncher =
registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { permissions ->
if (permissions[Manifest.permission.CAMERA] == true && permissions[Manifest.permission.RECORD_AUDIO] == true) {
viewModel.permissionGranted() // we will add this later
}
}
private val permissions = listOf(
Manifest.permission.CAMERA,
Manifest.permission.RECORD_AUDIO,
)
private fun requestPermission() {
when {
this.hasPermissions(permissions) -> viewModel.permissionGranted() // we will add this later
else -> requestPermissionLauncher.launch(permissions.toTypedArray())
}
}
private fun Context.hasPermissions(permissions: List): Boolean {
return permissions.all {
ContextCompat.checkSelfPermission(this, it) == PackageManager.PERMISSION_GRANTED
}
}
```
## App State
Our application keeps track of the participants locally in a `MainViewModel.kt` and the state will be communicated back to the `MainActivity` using Kotlin’s [StateFlow](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines.flow/-state-flow/).
Create a new Kotlin class `MainViewModel`:
```
package com.amazonaws.ivs.realtime.basicrealtime
import android.app.Application
import androidx.lifecycle.AndroidViewModel
class MainViewModel(application: Application) : AndroidViewModel(application), Stage.Strategy, StageRenderer {
}
```
In `MainActivity.kt` we manage our view model:
```
import androidx.activity.viewModels
private val viewModel: MainViewModel by viewModels()
```
To use `AndroidViewModel` and these Kotlin `ViewModel` extensions, you’ll need to add the following to your module’s `build.gradle` file:
```
implementation 'androidx.core:core-ktx:1.10.1'
implementation "androidx.activity:activity-ktx:1.7.2"
implementation 'androidx.appcompat:appcompat:1.6.1'
implementation 'com.google.android.material:material:1.10.0'
implementation "androidx.lifecycle:lifecycle-extensions:2.2.0"
def lifecycle_version = "2.6.1"
implementation "androidx.lifecycle:lifecycle-livedata-ktx:$lifecycle_version"
implementation "androidx.lifecycle:lifecycle-viewmodel-ktx:$lifecycle_version"
implementation 'androidx.constraintlayout:constraintlayout:2.1.4'
```
### RecyclerView Adapter
We’ll create a simple `RecyclerView.Adapter` subclass to keep track of our participants and update our `RecyclerView` on stage events. But first, we need a class that represents a participant. Create a new Kotlin class `StageParticipant`:
```
package com.amazonaws.ivs.realtime.basicrealtime
import com.amazonaws.ivs.broadcast.Stage
import com.amazonaws.ivs.broadcast.StageStream
class StageParticipant(val isLocal: Boolean, var participantId: String?) {
var publishState = Stage.PublishState.NOT_PUBLISHED
var subscribeState = Stage.SubscribeState.NOT_SUBSCRIBED
var streams = mutableListOf()
val stableID: String
get() {
return if (isLocal) {
"LocalUser"
} else {
requireNotNull(participantId)
}
}
}
```
We’ll use this class in the `ParticipantAdapter` class that we’ll create next. We start by defining the class and creating a variable to track the participants:
```
package com.amazonaws.ivs.realtime.basicrealtime
import android.view.LayoutInflater
import android.view.ViewGroup
import androidx.recyclerview.widget.RecyclerView
class ParticipantAdapter : RecyclerView.Adapter() {
private val participants = mutableListOf()
```
We also have to define our `RecyclerView.ViewHolder` before implementing the rest of the overrides:
```
class ViewHolder(val participantItem: ParticipantItem) : RecyclerView.ViewHolder(participantItem)
```
Using this, we can implement the standard `RecyclerView.Adapter` overrides:
```
override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): ViewHolder {
val item = LayoutInflater.from(parent.context)
.inflate(R.layout.item_stage_participant, parent, false) as ParticipantItem
return ViewHolder(item)
}
override fun getItemCount(): Int {
return participants.size
}
override fun getItemId(position: Int): Long =
participants[position]
.stableID
.hashCode()
.toLong()
override fun onBindViewHolder(holder: ViewHolder, position: Int) {
return holder.participantItem.bind(participants[position])
}
override fun onBindViewHolder(holder: ViewHolder, position: Int, payloads: MutableList) {
val updates = payloads.filterIsInstance()
if (updates.isNotEmpty()) {
updates.forEach { holder.participantItem.bind(it) // implemented later }
} else {
super.onBindViewHolder(holder, position, payloads)
}
}
```
Finally, we add new methods that we will call from our `MainViewModel` when changes to participants are made. These methods are standard CRUD operations on the adapter.
```
fun participantJoined(participant: StageParticipant) {
participants.add(participant)
notifyItemInserted(participants.size - 1)
}
fun participantLeft(participantId: String) {
val index = participants.indexOfFirst { it.participantId == participantId }
if (index != -1) {
participants.removeAt(index)
notifyItemRemoved(index)
}
}
fun participantUpdated(participantId: String?, update: (participant: StageParticipant) -> Unit) {
val index = participants.indexOfFirst { it.participantId == participantId }
if (index != -1) {
update(participants[index])
notifyItemChanged(index, participants[index])
}
}
```
Back in `MainViewModel` we need to create and hold a reference to this adapter:
```
internal val participantAdapter = ParticipantAdapter()
```
## Stage State
We also need to track some stage state within `MainViewModel`. Let’s define those properties now:
```
private val _connectionState = MutableStateFlow(Stage.ConnectionState.DISCONNECTED)
val connectionState = _connectionState.asStateFlow()
private var publishEnabled: Boolean = false
set(value) {
field = value
// Because the strategy returns the value of `checkboxPublish.isChecked`, just call `refreshStrategy`.
stage?.refreshStrategy()
}
private var deviceDiscovery: DeviceDiscovery? = null
private var stage: Stage? = null
private var streams = mutableListOf()
```
To see your own preview before joining a stage, we create a local participant immediately:
```
init {
deviceDiscovery = DeviceDiscovery(application)
// Create a local participant immediately to render our camera preview and microphone stats
val localParticipant = StageParticipant(true, null)
participantAdapter.participantJoined(localParticipant)
}
```
We want to make sure we clean up these resources when our `ViewModel` is cleaned up. We override `onCleared()` right away, so we don’t forget to clean these resources.
```
override fun onCleared() {
stage?.release()
deviceDiscovery?.release()
deviceDiscovery = null
super.onCleared()
}
```
Now we populate our local `streams` property as soon as permissions are granted, implementing the `permissionsGranted` method that we called earlier:
```
internal fun permissionGranted() {
val deviceDiscovery = deviceDiscovery ?: return
streams.clear()
val devices = deviceDiscovery.listLocalDevices()
// Camera
devices
.filter { it.descriptor.type == Device.Descriptor.DeviceType.CAMERA }
.maxByOrNull { it.descriptor.position == Device.Descriptor.Position.FRONT }
?.let { streams.add(ImageLocalStageStream(it)) }
// Microphone
devices
.filter { it.descriptor.type == Device.Descriptor.DeviceType.MICROPHONE }
.maxByOrNull { it.descriptor.isDefault }
?.let { streams.add(AudioLocalStageStream(it)) }
stage?.refreshStrategy()
// Update our local participant with these new streams
participantAdapter.participantUpdated(null) {
it.streams.clear()
it.streams.addAll(streams)
}
}
```
## Implementing the Stage SDK
Three core [concepts](android-publish-subscribe.md#android-publish-subscribe-concepts) underlie real-time functionality: stage, strategy, and renderer. The design goal is minimizing the amount of client-side logic necessary to build a working product.
### Stage.Strategy
Our `Stage.Strategy` implementation is simple:
```
override fun stageStreamsToPublishForParticipant(
stage: Stage,
participantInfo: ParticipantInfo
): MutableList {
// Return the camera and microphone to be published.
// This is only called if `shouldPublishFromParticipant` returns true.
return streams
}
override fun shouldPublishFromParticipant(stage: Stage, participantInfo: ParticipantInfo): Boolean {
return publishEnabled
}
override fun shouldSubscribeToParticipant(stage: Stage, participantInfo: ParticipantInfo): Stage.SubscribeType {
// Subscribe to both audio and video for all publishing participants.
return Stage.SubscribeType.AUDIO_VIDEO
}
```
To summarize, we publish based on our internal `publishEnabled` state, and if we publish we will publish the streams we collected earlier. Finally for this sample, we always subscribe to other participants, receiving both their audio and video.
### StageRenderer
The `StageRenderer` implementation also is fairly simple, though given the number of functions it contains quite a bit more code. The general approach in this renderer is to update our `ParticipantAdapter` when the SDK notifies us of a change to a participant. There are certain scenarios where we handle local participants differently, because we have decided to manage them ourselves so they can see their camera preview before joining.
```
override fun onError(exception: BroadcastException) {
Toast.makeText(getApplication(), "onError ${exception.localizedMessage}", Toast.LENGTH_LONG).show()
Log.e("BasicRealTime", "onError $exception")
}
override fun onConnectionStateChanged(
stage: Stage,
connectionState: Stage.ConnectionState,
exception: BroadcastException?
) {
_connectionState.value = connectionState
}
override fun onParticipantJoined(stage: Stage, participantInfo: ParticipantInfo) {
if (participantInfo.isLocal) {
// If this is the local participant joining the stage, update the participant with a null ID because we
// manually added that participant when setting up our preview
participantAdapter.participantUpdated(null) {
it.participantId = participantInfo.participantId
}
} else {
// If they are not local, add them normally
participantAdapter.participantJoined(
StageParticipant(
participantInfo.isLocal,
participantInfo.participantId
)
)
}
}
override fun onParticipantLeft(stage: Stage, participantInfo: ParticipantInfo) {
if (participantInfo.isLocal) {
// If this is the local participant leaving the stage, update the ID but keep it around because
// we want to keep the camera preview active
participantAdapter.participantUpdated(participantInfo.participantId) {
it.participantId = null
}
} else {
// If they are not local, have them leave normally
participantAdapter.participantLeft(participantInfo.participantId)
}
}
override fun onParticipantPublishStateChanged(
stage: Stage,
participantInfo: ParticipantInfo,
publishState: Stage.PublishState
) {
// Update the publishing state of this participant
participantAdapter.participantUpdated(participantInfo.participantId) {
it.publishState = publishState
}
}
override fun onParticipantSubscribeStateChanged(
stage: Stage,
participantInfo: ParticipantInfo,
subscribeState: Stage.SubscribeState
) {
// Update the subscribe state of this participant
participantAdapter.participantUpdated(participantInfo.participantId) {
it.subscribeState = subscribeState
}
}
override fun onStreamsAdded(stage: Stage, participantInfo: ParticipantInfo, streams: MutableList) {
// We don't want to take any action for the local participant because we track those streams locally
if (participantInfo.isLocal) {
return
}
// For remote participants, add these new streams to that participant's streams array.
participantAdapter.participantUpdated(participantInfo.participantId) {
it.streams.addAll(streams)
}
}
override fun onStreamsRemoved(stage: Stage, participantInfo: ParticipantInfo, streams: MutableList) {
// We don't want to take any action for the local participant because we track those streams locally
if (participantInfo.isLocal) {
return
}
// For remote participants, remove these streams from that participant's streams array.
participantAdapter.participantUpdated(participantInfo.participantId) {
it.streams.removeAll(streams)
}
}
override fun onStreamsMutedChanged(
stage: Stage,
participantInfo: ParticipantInfo,
streams: MutableList
) {
// We don't want to take any action for the local participant because we track those streams locally
if (participantInfo.isLocal) {
return
}
// For remote participants, notify the adapter that the participant has been updated. There is no need to modify
// the `streams` property on the `StageParticipant` because it is the same `StageStream` instance. Just
// query the `isMuted` property again.
participantAdapter.participantUpdated(participantInfo.participantId) {}
}
```
## Implementing a Custom RecyclerView LayoutManager
Laying out different numbers of participants can be complex. You want them to take up the entire parent view’s frame but you don’t want to handle each participant configuration independently. To make this easy, we’ll walk through implementing a `RecyclerView.LayoutManager`.
Create another new class, `StageLayoutManager`, which should extend `GridLayoutManager`. This class is designed to calculate the layout for each participant based on the number of participants in a flow-based row/column layout. Each row is the same height as the others, but columns can be different widths per row. See the code comment above the `layouts` variable for a description of how to customize this behavior.
```
package com.amazonaws.ivs.realtime.basicrealtime
import android.content.Context
import androidx.recyclerview.widget.GridLayoutManager
import androidx.recyclerview.widget.RecyclerView
class StageLayoutManager(context: Context?) : GridLayoutManager(context, 6) {
companion object {
/**
* This 2D array contains the description of how the grid of participants should be rendered
* The index of the 1st dimension is the number of participants needed to active that configuration
* Meaning if there is 1 participant, index 0 will be used. If there are 5 participants, index 4 will be used.
*
* The 2nd dimension is a description of the layout. The length of the array is the number of rows that
* will exist, and then each number within that array is the number of columns in each row.
*
* See the code comments next to each index for concrete examples.
*
* This can be customized to fit any layout configuration needed.
*/
val layouts: List> = listOf(
// 1 participant
listOf(1), // 1 row, full width
// 2 participants
listOf(1, 1), // 2 rows, all columns are full width
// 3 participants
listOf(1, 2), // 2 rows, first row's column is full width then 2nd row's columns are 1/2 width
// 4 participants
listOf(2, 2), // 2 rows, all columns are 1/2 width
// 5 participants
listOf(1, 2, 2), // 3 rows, first row's column is full width, 2nd and 3rd row's columns are 1/2 width
// 6 participants
listOf(2, 2, 2), // 3 rows, all column are 1/2 width
// 7 participants
listOf(2, 2, 3), // 3 rows, 1st and 2nd row's columns are 1/2 width, 3rd row's columns are 1/3rd width
// 8 participants
listOf(2, 3, 3),
// 9 participants
listOf(3, 3, 3),
// 10 participants
listOf(2, 3, 2, 3),
// 11 participants
listOf(2, 3, 3, 3),
// 12 participants
listOf(3, 3, 3, 3),
)
}
init {
spanSizeLookup = object : SpanSizeLookup() {
override fun getSpanSize(position: Int): Int {
if (itemCount <= 0) {
return 1
}
// Calculate the row we're in
val config = layouts[itemCount - 1]
var row = 0
var curPosition = position
while (curPosition - config[row] >= 0) {
curPosition -= config[row]
row++
}
// spanCount == max spans, config[row] = number of columns we want
// So spanCount / config[row] would be something like 6 / 3 if we want 3 columns.
// So this will take up 2 spans, with a max of 6 is 1/3rd of the view.
return spanCount / config[row]
}
}
}
override fun onLayoutChildren(recycler: RecyclerView.Recycler?, state: RecyclerView.State?) {
if (itemCount <= 0 || state?.isPreLayout == true) return
val parentHeight = height
val itemHeight = parentHeight / layouts[itemCount - 1].size // height divided by number of rows.
// Set the height of each view based on how many rows exist for the current participant count.
for (i in 0 until childCount) {
val child = getChildAt(i) ?: continue
val layoutParams = child.layoutParams as RecyclerView.LayoutParams
if (layoutParams.height != itemHeight) {
layoutParams.height = itemHeight
child.layoutParams = layoutParams
}
}
// After we set the height for all our views, call super.
// This works because our RecyclerView can not scroll and all views are always visible with stable IDs.
super.onLayoutChildren(recycler, state)
}
override fun canScrollVertically(): Boolean = false
override fun canScrollHorizontally(): Boolean = false
}
```
Back in `MainActivity.kt` we need to set the adapter and layout manager for our `RecyclerView`:
```
// In onCreate after setting recyclerView.
recyclerView.layoutManager = StageLayoutManager(this)
recyclerView.adapter = viewModel.participantAdapter
```
## Hooking Up UI Actions
We are getting close; there are just a few UI actions that we need to hook up.
First we’ll have our `MainActivity` observe the `StateFlow` changes from `MainViewModel`:
```
// At the end of your onCreate method
lifecycleScope.launch {
repeatOnLifecycle(Lifecycle.State.CREATED) {
viewModel.connectionState.collect { state ->
buttonJoin.setText(if (state == ConnectionState.DISCONNECTED) R.string.join else R.string.leave)
textViewState.text = getString(R.string.state, state.name)
}
}
}
```
Next we add listeners to our Join button and Publish checkbox:
```
buttonJoin.setOnClickListener {
viewModel.joinStage(editTextToken.text.toString())
}
checkboxPublish.setOnCheckedChangeListener { _, isChecked ->
viewModel.setPublishEnabled(isChecked)
}
```
Both of the above call functionality in our `MainViewModel`, which we implement now:
```
internal fun joinStage(token: String) {
if (_connectionState.value != Stage.ConnectionState.DISCONNECTED) {
// If we're already connected to a stage, leave it.
stage?.leave()
} else {
if (token.isEmpty()) {
Toast.makeText(getApplication(), "Empty Token", Toast.LENGTH_SHORT).show()
return
}
try {
// Destroy the old stage first before creating a new one.
stage?.release()
val stage = Stage(getApplication(), token, this)
stage.addRenderer(this)
stage.join()
this.stage = stage
} catch (e: BroadcastException) {
Toast.makeText(getApplication(), "Failed to join stage ${e.localizedMessage}", Toast.LENGTH_LONG).show()
e.printStackTrace()
}
}
}
internal fun setPublishEnabled(enabled: Boolean) {
publishEnabled = enabled
}
```
## Rendering the Participants
Finally, we need to render the data we receive from the SDK onto the participant item that we created earlier. We already have the `RecyclerView` logic finished, so we just need to implement the `bind` API in `ParticipantItem`.
We’ll start by adding the empty function and then walk through it step by step:
```
fun bind(participant: StageParticipant) {
}
```
First we’ll handle the easy state, the participant ID, publish state, and subscribe state. For these, we just update our `TextViews` directly:
```
val participantId = if (participant.isLocal) {
"You (${participant.participantId ?: "Disconnected"})"
} else {
participant.participantId
}
textViewParticipantId.text = participantId
textViewPublish.text = participant.publishState.name
textViewSubscribe.text = participant.subscribeState.name
```
Next we’ll update the audio and video muted states. To get the muted state, we need to find the `ImageDevice` and `AudioDevice` from the streams array. To optimize performance, we remember the last attached device IDs.
```
// This belongs outside the `bind` API.
private var imageDeviceUrn: String? = null
private var audioDeviceUrn: String? = null
// This belongs inside the `bind` API.
val newImageStream = participant
.streams
.firstOrNull { it.device is ImageDevice }
textViewVideoMuted.text = if (newImageStream != null) {
if (newImageStream.muted) "Video muted" else "Video not muted"
} else {
"No video stream"
}
val newAudioStream = participant
.streams
.firstOrNull { it.device is AudioDevice }
textViewAudioMuted.text = if (newAudioStream != null) {
if (newAudioStream.muted) "Audio muted" else "Audio not muted"
} else {
"No audio stream"
}
```
Finally we want to render a preview for the `imageDevice`:
```
if (newImageStream?.device?.descriptor?.urn != imageDeviceUrn) {
// If the device has changed, remove all subviews from the preview container
previewContainer.removeAllViews()
(newImageStream?.device as? ImageDevice)?.let {
val preview = it.getPreviewView(BroadcastConfiguration.AspectMode.FIT)
previewContainer.addView(preview)
preview.layoutParams = FrameLayout.LayoutParams(
FrameLayout.LayoutParams.MATCH_PARENT,
FrameLayout.LayoutParams.MATCH_PARENT
)
}
}
imageDeviceUrn = newImageStream?.device?.descriptor?.urn
```
And we display audio stats from the `audioDevice`:
```
if (newAudioStream?.device?.descriptor?.urn != audioDeviceUrn) {
(newAudioStream?.device as? AudioDevice)?.let {
it.setStatsCallback { _, rms ->
textViewAudioLevel.text = "Audio Level: ${rms.roundToInt()} dB"
}
}
}
audioDeviceUrn = newAudioStream?.device?.descriptor?.urn
```
# Publish & Subscribe with the IVS iOS Broadcast SDK
This section takes you through the steps involved in publishing and subscribing to a stage using your iOS app.
## Create Views
We start by using the auto-created `ViewController.swift` file to import `AmazonIVSBroadcast` and then add some `@IBOutlets` to link:
```
import AmazonIVSBroadcast
class ViewController: UIViewController {
@IBOutlet private var textFieldToken: UITextField!
@IBOutlet private var buttonJoin: UIButton!
@IBOutlet private var labelState: UILabel!
@IBOutlet private var switchPublish: UISwitch!
@IBOutlet private var collectionViewParticipants: UICollectionView!
```
Now we create those views and link them up in `Main.storyboard`. Here is the view structure that we’ll use:
![\[Use Main.storyboard to create an iOS view.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_1.png)
For AutoLayout configuration, we need to customize three views. The first view is **Collection View Participants** (a `UICollectionView`). Bound **Leading**, **Trailing**, and **Bottom** to **Safe Area**. Also bound **Top** to **Controls Container**.
![\[Customize iOS Collection View Participants view.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_2.png)
The second view is **Controls Container**. Bound **Leading**, **Trailing**, and **Top** to **Safe Area**:
![\[Customize iOS Controls Container view.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_3.png)
The third and last view is **Vertical Stack View**. Bound **Top**, **Leading**, **Trailing**, and **Bottom** to **Superview**. For styling, set the spacing to 8 instead of 0.
![\[Customize iOS Vertical Stack view.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_4.png)
The **UIStackViews** will handle the layout of the remaining views. For all three **UIStackViews**, use **Fill** as the **Alignment** and **Distribution**.
![\[Customize remaining iOS views with UIStackViews.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_5.png)
Finally, let’s link these views to our `ViewController`. From above, map the following views:
+ **Text Field Join** binds to `textFieldToken`.
+ **Button Join** binds to `buttonJoin`.
+ **Label State** binds to `labelState`.
+ **Switch Publish** binds to `switchPublish`.
+ **Collection View Participants** binds to `collectionViewParticipants`.
Also use this time to set the `dataSource` of the **Collection View Participants** item to the owning `ViewController`:
![\[Set the dataSource of Collection View Participants for iOS app.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_6.png)
Now we create the `UICollectionViewCell` subclass in which to render the participants. Start by creating a new **Cocoa Touch Class** file:
![\[Create a UICollectionViewCell to render iOS real-time participants.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_7.png)
Name it `ParticipantUICollectionViewCell` and make it a subclass of `UICollectionViewCell` in Swift. We start in the Swift file again, creating our `@IBOutlets` to link:
```
import AmazonIVSBroadcast
class ParticipantCollectionViewCell: UICollectionViewCell {
@IBOutlet private var viewPreviewContainer: UIView!
@IBOutlet private var labelParticipantId: UILabel!
@IBOutlet private var labelSubscribeState: UILabel!
@IBOutlet private var labelPublishState: UILabel!
@IBOutlet private var labelVideoMuted: UILabel!
@IBOutlet private var labelAudioMuted: UILabel!
@IBOutlet private var labelAudioVolume: UILabel!
```
In the associated XIB file, create this view hierarchy:
![\[Create iOS view hierarchy in associated XIB file.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_8.png)
For AutoLayout, we’ll modify three views again. The first view is **View Preview Container**. Set **Trailing**, **Leading**, **Top**, and **Bottom** to **Participant Collection View Cell**.
![\[Customize iOS View Preview Container view.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_9.png)
The second view is **View**. Set **Leading** and **Top** to **Participant Collection View Cell** and change the value to 4.
![\[Customize iOS View view.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_10.png)
The third view is **Stack View**. Set **Trailing**, **Leading**, **Top**, and **Bottom** to **Superview** and change the value to 4.
![\[Customize iOS Stack View view.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_11.png)
## Permissions and Idle Timer
Going back to our `ViewController`, we will disable the system idle timer to prevent the device from going to sleep while our application is being used:
```
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
// Prevent the screen from turning off during a call.
UIApplication.shared.isIdleTimerDisabled = true
}
override func viewDidDisappear(_ animated: Bool) {
super.viewDidDisappear(animated)
UIApplication.shared.isIdleTimerDisabled = false
}
```
Next we request camera and microphone permissions from the system:
```
private func checkPermissions() {
checkOrGetPermission(for: .video) { [weak self] granted in
guard granted else {
print("Video permission denied")
return
}
self?.checkOrGetPermission(for: .audio) { [weak self] granted in
guard granted else {
print("Audio permission denied")
return
}
self?.setupLocalUser() // we will cover this later
}
}
}
private func checkOrGetPermission(for mediaType: AVMediaType, _ result: @escaping (Bool) -> Void) {
func mainThreadResult(_ success: Bool) {
DispatchQueue.main.async {
result(success)
}
}
switch AVCaptureDevice.authorizationStatus(for: mediaType) {
case .authorized: mainThreadResult(true)
case .notDetermined:
AVCaptureDevice.requestAccess(for: mediaType) { granted in
mainThreadResult(granted)
}
case .denied, .restricted: mainThreadResult(false)
@unknown default: mainThreadResult(false)
}
}
```
## App State
We need to configure our `collectionViewParticipants` with the layout file that we created earlier:
```
override func viewDidLoad() {
super.viewDidLoad()
// We render everything to exactly the frame, so don't allow scrolling.
collectionViewParticipants.isScrollEnabled = false
collectionViewParticipants.register(UINib(nibName: "ParticipantCollectionViewCell", bundle: .main), forCellWithReuseIdentifier: "ParticipantCollectionViewCell")
}
```
To represent each participant, we create a simple struct called `StageParticipant`. This can be included in the `ViewController.swift` file, or a new file can be created.
```
import Foundation
import AmazonIVSBroadcast
struct StageParticipant {
let isLocal: Bool
var participantId: String?
var publishState: IVSParticipantPublishState = .notPublished
var subscribeState: IVSParticipantSubscribeState = .notSubscribed
var streams: [IVSStageStream] = []
init(isLocal: Bool, participantId: String?) {
self.isLocal = isLocal
self.participantId = participantId
}
}
```
To track those participants, we keep an array of them as a private property in our `ViewController`:
```
private var participants = [StageParticipant]()
```
This property will be used to power our `UICollectionViewDataSource` that was linked from the storyboard earlier:
```
extension ViewController: UICollectionViewDataSource {
func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
return participants.count
}
func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
if let cell = collectionView.dequeueReusableCell(withReuseIdentifier: "ParticipantCollectionViewCell", for: indexPath) as? ParticipantCollectionViewCell {
cell.set(participant: participants[indexPath.row])
return cell
} else {
fatalError("Couldn't load custom cell type 'ParticipantCollectionViewCell'")
}
}
}
```
To see your own preview before joining a stage, we create a local participant immediately:
```
override func viewDidLoad() {
/* existing UICollectionView code */
participants.append(StageParticipant(isLocal: true, participantId: nil))
}
```
This results in a participant cell being rendered immediately once the app is running, representing the local participant.
Users want to be able to see themselves before joining a stage, so next we implement the `setupLocalUser()` method that gets called from the permissions-handling code earlier. We store the camera and microphone reference as `IVSLocalStageStream` objects.
```
private var streams = [IVSLocalStageStream]()
private let deviceDiscovery = IVSDeviceDiscovery()
private func setupLocalUser() {
// Gather our camera and microphone once permissions have been granted
let devices = deviceDiscovery.listLocalDevices()
streams.removeAll()
if let camera = devices.compactMap({ $0 as? IVSCamera }).first {
streams.append(IVSLocalStageStream(device: camera))
// Use a front camera if available.
if let frontSource = camera.listAvailableInputSources().first(where: { $0.position == .front }) {
camera.setPreferredInputSource(frontSource)
}
}
if let mic = devices.compactMap({ $0 as? IVSMicrophone }).first {
streams.append(IVSLocalStageStream(device: mic))
}
participants[0].streams = streams
participantsChanged(index: 0, changeType: .updated)
}
```
Here we’ve found the device’s camera and microphone through the SDK and stored them in our local `streams` object, then assigned the `streams` array of the first participant (the local participant that we created earlier) to our `streams`. Finally we call `participantsChanged` with an `index` of 0 and `changeType` of `updated`. That function is a helper function for updating our `UICollectionView` with nice animations. Here’s what it looks like:
```
private func participantsChanged(index: Int, changeType: ChangeType) {
switch changeType {
case .joined:
collectionViewParticipants?.insertItems(at: [IndexPath(item: index, section: 0)])
case .updated:
// Instead of doing reloadItems, just grab the cell and update it ourselves. It saves a create/destroy of a cell
// and more importantly fixes some UI flicker. We disable scrolling so the index path per cell
// never changes.
if let cell = collectionViewParticipants?.cellForItem(at: IndexPath(item: index, section: 0)) as? ParticipantCollectionViewCell {
cell.set(participant: participants[index])
}
case .left:
collectionViewParticipants?.deleteItems(at: [IndexPath(item: index, section: 0)])
}
}
```
Don’t worry about `cell.set` yet; we’ll get to that later, but that’s where we will render the cell’s contents based on the participant.
The `ChangeType` is a simple enum:
```
enum ChangeType {
case joined, updated, left
}
```
Finally, we want to keep track of whether the stage is connected. We use a simple `bool` to track that, which will automatically update our UI when it is updated itself.
```
private var connectingOrConnected = false {
didSet {
buttonJoin.setTitle(connectingOrConnected ? "Leave" : "Join", for: .normal)
buttonJoin.tintColor = connectingOrConnected ? .systemRed : .systemBlue
}
}
```
## Implement the Stage SDK
Three core [concepts](ios-publish-subscribe.md#ios-publish-subscribe-concepts) underlie real-time functionality: stage, strategy, and renderer. The design goal is minimizing the amount of client-side logic necessary to build a working product.
### IVSStageStrategy
Our `IVSStageStrategy` implementation is simple:
```
extension ViewController: IVSStageStrategy {
func stage(_ stage: IVSStage, streamsToPublishForParticipant participant: IVSParticipantInfo) -> [IVSLocalStageStream] {
// Return the camera and microphone to be published.
// This is only called if `shouldPublishParticipant` returns true.
return streams
}
func stage(_ stage: IVSStage, shouldPublishParticipant participant: IVSParticipantInfo) -> Bool {
// Our publish status is based directly on the UISwitch view
return switchPublish.isOn
}
func stage(_ stage: IVSStage, shouldSubscribeToParticipant participant: IVSParticipantInfo) -> IVSStageSubscribeType {
// Subscribe to both audio and video for all publishing participants.
return .audioVideo
}
}
```
To summarize, we only publish if the publish switch is in the “on” position, and if we publish we will publish the streams that we collected earlier. Finally, for this sample, we always subscribe to other participants, receiving both their audio and video.
### IVSStageRenderer
The `IVSStageRenderer` implementation also is fairly simple, though given the number of functions it contains quite a bit more code. The general approach in this renderer is to update our `participants` array when the SDK notifies us of a change to a participant. There are certain scenarios where we handle local participants differently, because we have decided to manage them ourselves so they can see their camera preview before joining.
```
extension ViewController: IVSStageRenderer {
func stage(_ stage: IVSStage, didChange connectionState: IVSStageConnectionState, withError error: Error?) {
labelState.text = connectionState.text
connectingOrConnected = connectionState != .disconnected
}
func stage(_ stage: IVSStage, participantDidJoin participant: IVSParticipantInfo) {
if participant.isLocal {
// If this is the local participant joining the Stage, update the first participant in our array because we
// manually added that participant when setting up our preview
participants[0].participantId = participant.participantId
participantsChanged(index: 0, changeType: .updated)
} else {
// If they are not local, add them to the array as a newly joined participant.
participants.append(StageParticipant(isLocal: false, participantId: participant.participantId))
participantsChanged(index: (participants.count - 1), changeType: .joined)
}
}
func stage(_ stage: IVSStage, participantDidLeave participant: IVSParticipantInfo) {
if participant.isLocal {
// If this is the local participant leaving the Stage, update the first participant in our array because
// we want to keep the camera preview active
participants[0].participantId = nil
participantsChanged(index: 0, changeType: .updated)
} else {
// If they are not local, find their index and remove them from the array.
if let index = participants.firstIndex(where: { $0.participantId == participant.participantId }) {
participants.remove(at: index)
participantsChanged(index: index, changeType: .left)
}
}
}
func stage(_ stage: IVSStage, participant: IVSParticipantInfo, didChange publishState: IVSParticipantPublishState) {
// Update the publishing state of this participant
mutatingParticipant(participant.participantId) { data in
data.publishState = publishState
}
}
func stage(_ stage: IVSStage, participant: IVSParticipantInfo, didChange subscribeState: IVSParticipantSubscribeState) {
// Update the subscribe state of this participant
mutatingParticipant(participant.participantId) { data in
data.subscribeState = subscribeState
}
}
func stage(_ stage: IVSStage, participant: IVSParticipantInfo, didChangeMutedStreams streams: [IVSStageStream]) {
// We don't want to take any action for the local participant because we track those streams locally
if participant.isLocal { return }
// For remote participants, notify the UICollectionView that they have updated. There is no need to modify
// the `streams` property on the `StageParticipant` because it is the same `IVSStageStream` instance. Just
// query the `isMuted` property again.
if let index = participants.firstIndex(where: { $0.participantId == participant.participantId }) {
participantsChanged(index: index, changeType: .updated)
}
}
func stage(_ stage: IVSStage, participant: IVSParticipantInfo, didAdd streams: [IVSStageStream]) {
// We don't want to take any action for the local participant because we track those streams locally
if participant.isLocal { return }
// For remote participants, add these new streams to that participant's streams array.
mutatingParticipant(participant.participantId) { data in
data.streams.append(contentsOf: streams)
}
}
func stage(_ stage: IVSStage, participant: IVSParticipantInfo, didRemove streams: [IVSStageStream]) {
// We don't want to take any action for the local participant because we track those streams locally
if participant.isLocal { return }
// For remote participants, remove these streams from that participant's streams array.
mutatingParticipant(participant.participantId) { data in
let oldUrns = streams.map { $0.device.descriptor().urn }
data.streams.removeAll(where: { stream in
return oldUrns.contains(stream.device.descriptor().urn)
})
}
}
// A helper function to find a participant by its ID, mutate that participant, and then update the UICollectionView accordingly.
private func mutatingParticipant(_ participantId: String?, modifier: (inout StageParticipant) -> Void) {
guard let index = participants.firstIndex(where: { $0.participantId == participantId }) else {
fatalError("Something is out of sync, investigate if this was a sample app or SDK issue.")
}
var participant = participants[index]
modifier(&participant)
participants[index] = participant
participantsChanged(index: index, changeType: .updated)
}
}
```
This code uses an extension to convert the connection state into human-friendly text:
```
extension IVSStageConnectionState {
var text: String {
switch self {
case .disconnected: return "Disconnected"
case .connecting: return "Connecting"
case .connected: return "Connected"
@unknown default: fatalError()
}
}
}
```
## Implementing a Custom UICollectionViewLayout
Laying out different numbers of participants can be complex. You want them to take up the entire parent view’s frame but you don’t want to handle each participant configuration independently. To make this easy, we’ll walk through implementing a `UICollectionViewLayout`.
Create another new file, `ParticipantCollectionViewLayout.swift`, which should extend `UICollectionViewLayout`. This class will use another class called `StageLayoutCalculator`, which we’ll cover soon. The class receives calculated frame values for each participant and then generates the necessary `UICollectionViewLayoutAttributes` objects.
```
import Foundation
import UIKit
/**
Code modified from https://developer.apple.com/documentation/uikit/views_and_controls/collection_views/layouts/customizing_collection_view_layouts?language=objc
*/
class ParticipantCollectionViewLayout: UICollectionViewLayout {
private let layoutCalculator = StageLayoutCalculator()
private var contentBounds = CGRect.zero
private var cachedAttributes = [UICollectionViewLayoutAttributes]()
override func prepare() {
super.prepare()
guard let collectionView = collectionView else { return }
cachedAttributes.removeAll()
contentBounds = CGRect(origin: .zero, size: collectionView.bounds.size)
layoutCalculator.calculateFrames(participantCount: collectionView.numberOfItems(inSection: 0),
width: collectionView.bounds.size.width,
height: collectionView.bounds.size.height,
padding: 4)
.enumerated()
.forEach { (index, frame) in
let attributes = UICollectionViewLayoutAttributes(forCellWith: IndexPath(item: index, section: 0))
attributes.frame = frame
cachedAttributes.append(attributes)
contentBounds = contentBounds.union(frame)
}
}
override var collectionViewContentSize: CGSize {
return contentBounds.size
}
override func shouldInvalidateLayout(forBoundsChange newBounds: CGRect) -> Bool {
guard let collectionView = collectionView else { return false }
return !newBounds.size.equalTo(collectionView.bounds.size)
}
override func layoutAttributesForItem(at indexPath: IndexPath) -> UICollectionViewLayoutAttributes? {
return cachedAttributes[indexPath.item]
}
override func layoutAttributesForElements(in rect: CGRect) -> [UICollectionViewLayoutAttributes]? {
var attributesArray = [UICollectionViewLayoutAttributes]()
// Find any cell that sits within the query rect.
guard let lastIndex = cachedAttributes.indices.last, let firstMatchIndex = binSearch(rect, start: 0, end: lastIndex) else {
return attributesArray
}
// Starting from the match, loop up and down through the array until all the attributes
// have been added within the query rect.
for attributes in cachedAttributes[..= rect.minY else { break }
attributesArray.append(attributes)
}
for attributes in cachedAttributes[firstMatchIndex...] {
guard attributes.frame.minY <= rect.maxY else { break }
attributesArray.append(attributes)
}
return attributesArray
}
// Perform a binary search on the cached attributes array.
func binSearch(_ rect: CGRect, start: Int, end: Int) -> Int? {
if end < start { return nil }
let mid = (start + end) / 2
let attr = cachedAttributes[mid]
if attr.frame.intersects(rect) {
return mid
} else {
if attr.frame.maxY < rect.minY {
return binSearch(rect, start: (mid + 1), end: end)
} else {
return binSearch(rect, start: start, end: (mid - 1))
}
}
}
}
```
More important is the `StageLayoutCalculator.swift` class. It is designed to calculate the frames for each participant based on the number of participants in a flow-based row/column layout. Each row is the same height as the others, but the columns can be different widths per row. See the code comment above the `layouts` variable for a description of how to customize this behavior.
```
import Foundation
import UIKit
class StageLayoutCalculator {
/// This 2D array contains the description of how the grid of participants should be rendered
/// The index of the 1st dimension is the number of participants needed to active that configuration
/// Meaning if there is 1 participant, index 0 will be used. If there are 5 participants, index 4 will be used.
///
/// The 2nd dimension is a description of the layout. The length of the array is the number of rows that
/// will exist, and then each number within that array is the number of columns in each row.
///
/// See the code comments next to each index for concrete examples.
///
/// This can be customized to fit any layout configuration needed.
private let layouts: [[Int]] = [
// 1 participant
[ 1 ], // 1 row, full width
// 2 participants
[ 1, 1 ], // 2 rows, all columns are full width
// 3 participants
[ 1, 2 ], // 2 rows, first row's column is full width then 2nd row's columns are 1/2 width
// 4 participants
[ 2, 2 ], // 2 rows, all columns are 1/2 width
// 5 participants
[ 1, 2, 2 ], // 3 rows, first row's column is full width, 2nd and 3rd row's columns are 1/2 width
// 6 participants
[ 2, 2, 2 ], // 3 rows, all column are 1/2 width
// 7 participants
[ 2, 2, 3 ], // 3 rows, 1st and 2nd row's columns are 1/2 width, 3rd row's columns are 1/3rd width
// 8 participants
[ 2, 3, 3 ],
// 9 participants
[ 3, 3, 3 ],
// 10 participants
[ 2, 3, 2, 3 ],
// 11 participants
[ 2, 3, 3, 3 ],
// 12 participants
[ 3, 3, 3, 3 ],
]
// Given a frame (this could be for a UICollectionView, or a Broadcast Mixer's canvas), calculate the frames for each
// participant, with optional padding.
func calculateFrames(participantCount: Int, width: CGFloat, height: CGFloat, padding: CGFloat) -> [CGRect] {
if participantCount > layouts.count {
fatalError("Only \(layouts.count) participants are supported at this time")
}
if participantCount == 0 {
return []
}
var currentIndex = 0
var lastFrame: CGRect = .zero
// If the height is less than the width, the rows and columns will be flipped.
// Meaning for 6 participants, there will be 2 rows of 3 columns each.
let isVertical = height > width
let halfPadding = padding / 2.0
let layout = layouts[participantCount - 1] // 1 participant is in index 0, so `-1`.
let rowHeight = (isVertical ? height : width) / CGFloat(layout.count)
var frames = [CGRect]()
for row in 0 ..< layout.count {
// layout[row] is the number of columns in a layout
let itemWidth = (isVertical ? width : height) / CGFloat(layout[row])
let segmentFrame = CGRect(x: (isVertical ? 0 : lastFrame.maxX) + halfPadding,
y: (isVertical ? lastFrame.maxY : 0) + halfPadding,
width: (isVertical ? itemWidth : rowHeight) - padding,
height: (isVertical ? rowHeight : itemWidth) - padding)
for column in 0 ..< layout[row] {
var frame = segmentFrame
if isVertical {
frame.origin.x = (itemWidth * CGFloat(column)) + halfPadding
} else {
frame.origin.y = (itemWidth * CGFloat(column)) + halfPadding
}
frames.append(frame)
currentIndex += 1
}
lastFrame = segmentFrame
lastFrame.origin.x += halfPadding
lastFrame.origin.y += halfPadding
}
return frames
}
}
```
Back in `Main.storyboard`, be sure to set the layout class for the `UICollectionView` to the class we just created:
![\[Xcode interface showing storyboard with UICollectionView and its layout settings.\]](http://docs.aws.amazon.com/ivs/latest/RealTimeUserGuide/images/Publish_iOS_12.png)
## Hooking Up UI Actions
We are getting close, there are a few `IBActions` that we need to create.
First we’ll handle the join button. It responds differently based on the value of `connectingOrConnected`. When it is already connected, it just leaves the stage. If it is disconnected, it reads the text from the token `UITextField` and creates a new `IVSStage` with that text. Then we add our `ViewController` as the `strategy`, `errorDelegate`, and renderer for the `IVSStage`, and finally we join the stage asynchronously.
```
@IBAction private func joinTapped(_ sender: UIButton) {
if connectingOrConnected {
// If we're already connected to a Stage, leave it.
stage?.leave()
} else {
guard let token = textFieldToken.text else {
print("No token")
return
}
// Hide the keyboard after tapping Join
textFieldToken.resignFirstResponder()
do {
// Destroy the old Stage first before creating a new one.
self.stage = nil
let stage = try IVSStage(token: token, strategy: self)
stage.errorDelegate = self
stage.addRenderer(self)
try stage.join()
self.stage = stage
} catch {
print("Failed to join stage - \(error)")
}
}
}
```
The other UI action we need to hook up is the publish switch:
```
@IBAction private func publishToggled(_ sender: UISwitch) {
// Because the strategy returns the value of `switchPublish.isOn`, just call `refreshStrategy`.
stage?.refreshStrategy()
}
```
## Rendering the Participants
Finally, we need to render the data we receive from the SDK onto the participant cell that we created earlier. We already have the `UICollectionView` logic finished, so we just need to implement the `set` API in `ParticipantCollectionViewCell.swift`.
We’ll start by adding the `empty` function and then walk through it step by step:
```
func set(participant: StageParticipant) {
}
```
First we handle the easy state, the participant ID, publish state, and subscribe state. For these, we just update our `UILabels` directly:
```
labelParticipantId.text = participant.isLocal ? "You (\(participant.participantId ?? "Disconnected"))" : participant.participantId
labelPublishState.text = participant.publishState.text
labelSubscribeState.text = participant.subscribeState.text
```
The text properties of the publish and subscribe enums come from local extensions:
```
extension IVSParticipantPublishState {
var text: String {
switch self {
case .notPublished: return "Not Published"
case .attemptingPublish: return "Attempting to Publish"
case .published: return "Published"
@unknown default: fatalError()
}
}
}
extension IVSParticipantSubscribeState {
var text: String {
switch self {
case .notSubscribed: return "Not Subscribed"
case .attemptingSubscribe: return "Attempting to Subscribe"
case .subscribed: return "Subscribed"
@unknown default: fatalError()
}
}
}
```
Next we update the audio and video muted states. To get the muted states we need to find the `IVSImageDevice` and `IVSAudioDevice` from the `streams` array. To optimize performance, we will remember the last devices attached.
```
// This belongs outside `set(participant:)`
private var registeredStreams: Set = []
private var imageDevice: IVSImageDevice? {
return registeredStreams.lazy.compactMap { $0.device as? IVSImageDevice }.first
}
private var audioDevice: IVSAudioDevice? {
return registeredStreams.lazy.compactMap { $0.device as? IVSAudioDevice }.first
}
// This belongs inside `set(participant:)`
let existingAudioStream = registeredStreams.first { $0.device is IVSAudioDevice }
let existingImageStream = registeredStreams.first { $0.device is IVSImageDevice }
registeredStreams = Set(participant.streams)
let newAudioStream = participant.streams.first { $0.device is IVSAudioDevice }
let newImageStream = participant.streams.first { $0.device is IVSImageDevice }
// `isMuted != false` covers the stream not existing, as well as being muted.
labelVideoMuted.text = "Video Muted: \(newImageStream?.isMuted != false)"
labelAudioMuted.text = "Audio Muted: \(newAudioStream?.isMuted != false)"
```
Finally we want to render a preview for the `imageDevice` and display audio stats from the `audioDevice`:
```
if existingImageStream !== newImageStream {
// The image stream has changed
updatePreview() // We’ll cover this next
}
if existingAudioStream !== newAudioStream {
(existingAudioStream?.device as? IVSAudioDevice)?.setStatsCallback(nil)
audioDevice?.setStatsCallback( { [weak self] stats in
self?.labelAudioVolume.text = String(format: "Audio Level: %.0f dB", stats.rms)
})
// When the audio stream changes, it will take some time to receive new stats. Reset the value temporarily.
self.labelAudioVolume.text = "Audio Level: -100 dB"
}
```
The last function we need to create is `updatePreview()`, which adds a preview of the participant to our view:
```
private func updatePreview() {
// Remove any old previews from the preview container
viewPreviewContainer.subviews.forEach { $0.removeFromSuperview() }
if let imageDevice = self.imageDevice {
if let preview = try? imageDevice.previewView(with: .fit) {
viewPreviewContainer.addSubviewMatchFrame(preview)
}
}
}
```
The above uses a helper function on `UIView` to make embedding subviews easier:
```
extension UIView {
func addSubviewMatchFrame(_ view: UIView) {
view.translatesAutoresizingMaskIntoConstraints = false
self.addSubview(view)
NSLayoutConstraint.activate([
view.topAnchor.constraint(equalTo: self.topAnchor, constant: 0),
view.bottomAnchor.constraint(equalTo: self.bottomAnchor, constant: 0),
view.leadingAnchor.constraint(equalTo: self.leadingAnchor, constant: 0),
view.trailingAnchor.constraint(equalTo: self.trailingAnchor, constant: 0),
])
}
}
```