Use stream manager in custom Greengrass components to store, process, and export IoT device data. Use the procedures and examples in this section to create component recipes, artifacts, and applications that work with stream manager. For more information about how to develop and test components, see Create AWS IoT Greengrass components.
Define component recipes that use stream manager
To use stream manager in a custom component, you must define the
aws.greengrass.StreamManager component as a dependency. You must also provide the
Stream Manager SDK. Complete the following tasks to download and use the Stream Manager SDK in the language of your
choice.
The Stream Manager SDK for Java is available as a JAR file that you can use to compile your component. Then, you can create an application JAR that includes the Stream Manager SDK, define the application JAR as a component artifact, and run the application JAR in the component lifecycle.
To use the Stream Manager SDK for Java
-
Download the Stream Manager SDK for Java JAR file
. -
Do one of the following to create component artifacts from your Java application and the Stream Manager SDK JAR file:
-
Build your application as a JAR file that includes the Stream Manager SDK JAR, and run this JAR file in your component recipe.
-
Define the Stream Manager SDK JAR as a component artifact. Add that artifact to the classpath when you run your application in your component recipe.
Your component recipe might look like the following example. This component runs a modified version of the StreamManagerS3.java
example, where StreamManagerS3.jarincludes the Stream Manager SDK JAR.For more information about how to develop and test components, see Create AWS IoT Greengrass components.
-
The Stream Manager SDK for Python is available as source code that you can include in your component. Create a ZIP file of the Stream Manager SDK, define the ZIP file as a component artifact, and install the SDK's requirements in the component lifecycle.
To use the Stream Manager SDK for Python
-
Clone or download the aws-greengrass-stream-manager-sdk-python
repository. git clone git@github.com:aws-greengrass/aws-greengrass-stream-manager-sdk-python.git -
Create a ZIP file that contains the
stream_managerfolder, which contains the source code of the Stream Manager SDK for Python. You can provide this ZIP file as a component artifact that the AWS IoT Greengrass Core software unzips when it installs your component. Do the following:-
Open the folder that contains the repository that you cloned or downloaded in the previous step.
cd aws-greengrass-stream-manager-sdk-python -
Zip the
stream_managerfolder into a ZIP file namedstream_manager_sdk.zip. -
Verify that the
stream_manager_sdk.zipfile contains thestream_managerfolder and its contents. Run the following command to list the contents of the ZIP file.The output should look similar to the following.
Archive: aws-greengrass-stream-manager-sdk-python/stream_manager.zip Length Date Time Name --------- ---------- ----- ---- 0 02-24-2021 20:45 stream_manager/ 913 02-24-2021 20:45 stream_manager/__init__.py 9719 02-24-2021 20:45 stream_manager/utilinternal.py 1412 02-24-2021 20:45 stream_manager/exceptions.py 1004 02-24-2021 20:45 stream_manager/util.py 0 02-24-2021 20:45 stream_manager/data/ 254463 02-24-2021 20:45 stream_manager/data/__init__.py 26515 02-24-2021 20:45 stream_manager/streammanagerclient.py --------- ------- 294026 8 files
-
-
Copy the Stream Manager SDK artifacts to your component's artifacts folder. In addition to the Stream Manager SDK ZIP file, your component uses the SDK's
requirements.txtfile to install the dependencies of the Stream Manager SDK. Replace~/greengrass-componentswith the path to the folder that you use for local development. -
Create your component recipe. In the recipe, do the following:
-
Define
stream_manager_sdk.zipandrequirements.txtas artifacts. -
Define your Python application as an artifact.
-
In the install lifecycle, install the Stream Manager SDK requirements from
requirements.txt. -
In the run lifecycle, append the Stream Manager SDK to
PYTHONPATH, and run your Python application.
Your component recipe might look like the following example. This component runs the stream_manager_s3.py
example. For more information about how to develop and test components, see Create AWS IoT Greengrass components.
-
The Stream Manager SDK for JavaScript is available as source code that you can include in your component. Create a ZIP file of the Stream Manager SDK, define the ZIP file as a component artifact, and install the SDK in the component lifecycle.
To use the Stream Manager SDK for JavaScript
-
Clone or download the aws-greengrass-stream-manager-sdk-js
repository. git clone git@github.com:aws-greengrass/aws-greengrass-stream-manager-sdk-js.git -
Create a ZIP file that contains the
aws-greengrass-stream-manager-sdkfolder, which contains the source code of the Stream Manager SDK for JavaScript. You can provide this ZIP file as a component artifact that the AWS IoT Greengrass Core software unzips when it installs your component. Do the following:-
Open the folder that contains the repository that you cloned or downloaded in the previous step.
cd aws-greengrass-stream-manager-sdk-js -
Zip the
aws-greengrass-stream-manager-sdkfolder into a ZIP file namedstream-manager-sdk.zip. -
Verify that the
stream-manager-sdk.zipfile contains theaws-greengrass-stream-manager-sdkfolder and its contents. Run the following command to list the contents of the ZIP file.The output should look similar to the following.
Archive: stream-manager-sdk.zip Length Date Time Name --------- ---------- ----- ---- 0 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/ 369 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/package.json 1017 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/util.js 8374 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/utilInternal.js 1937 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/exceptions.js 0 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/data/ 353343 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/data/index.js 22599 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/client.js 216 02-24-2021 22:36 aws-greengrass-stream-manager-sdk/index.js --------- ------- 387855 9 files
-
-
Copy the Stream Manager SDK artifact to your component's artifacts folder. Replace
~/greengrass-componentswith the path to the folder that you use for local development. -
Create your component recipe. In the recipe, do the following:
-
Define
stream-manager-sdk.zipas an artifact. -
Define your JavaScript application as an artifact.
-
In the install lifecycle, install the Stream Manager SDK from the
stream-manager-sdk.zipartifact. Thisnpm installcommand creates anode_modulesfolder that contains the Stream Manager SDK and its dependencies. -
In the run lifecycle, append the
node_modulesfolder toNODE_PATH, and run your JavaScript application.
Your component recipe might look like the following example. This component runs the StreamManagerS3
example. For more information about how to develop and test components, see Create AWS IoT Greengrass components.
-
Connect to stream manager in application code
To connect to stream manager in your application, create an instance of
StreamManagerClient from the Stream Manager SDK. This client connects to the stream
manager component on its default port 8088, or the port that you specify. For more information
about how to use StreamManagerClient after you create an instance, see Use StreamManagerClient to work with streams.
Example: Connect to stream manager with default port
Example: Connect to stream manager with non-default port
If you configure stream manager with a port other than the default, you must use interprocess communication to retrieve the port from the component configuration.
Note
The port configuration parameter contains the value that you specify in
STREAM_MANAGER_SERVER_PORT when you deploy stream manager.
void connectToStreamManagerWithCustomPort() {
EventStreamRPCConnection eventStreamRpcConnection = IPCUtils.getEventStreamRpcConnection();
GreengrassCoreIPCClient greengrassCoreIPCClient = new GreengrassCoreIPCClient(eventStreamRpcConnection);
List<String> keyPath = new ArrayList<>();
keyPath.add("port");
GetConfigurationRequest request = new GetConfigurationRequest();
request.setComponentName("aws.greengrass.StreamManager");
request.setKeyPath(keyPath);
GetConfigurationResponse response =
greengrassCoreIPCClient.getConfiguration(request, Optional.empty()).getResponse().get();
String port = response.getValue().get("port").toString();
System.out.print("Stream Manager is running on port: " + port);
final StreamManagerClientConfig config = StreamManagerClientConfig.builder()
.serverInfo(StreamManagerServerInfo.builder().port(Integer.parseInt(port)).build()).build();
StreamManagerClient client = StreamManagerClientFactory.standard().withClientConfig(config).build();
// Use the client.
}