This Guidance uses CloudWatch, CloudTrail, and AWS Config to improve monitoring, helping provide the information and alerts you need to respond quickly to events and facilitate compliance with strict requirements.
Overview
This Guidance demonstrates two ways that you can generate measurable value from research data, also known as data monetization. The first option guides you through a way to make data available to other AWS Cloud customers. With the second option, you can make your data available to customers who use other cloud providers or on-premises services. By monetizing data on AWS, either by a pay-per-use or a subscription model, you can diversify your revenue streams so that you don’t have to depend on someone else to sustainably share your research data. You can also set up this solution so that publishers or other stakeholders can set rules for content pricing based on your own parameters.
How it works
Research Data Monetization
This architecture diagram shows how you can improve data monetization with other AWS customers.
Download the architecture diagram PDF
Step 1
This is the entry point into the AWS environment. Ingest data from relational databases Amazon RDS, object stores like Amazon S3, NoSQL stores like Amazon DynamoDB, data lakes through Lake Formation, or external APIs through API Gateway. Lambda functions can be used for custom ingestion logic.
Step 2
AWS Glue serves as the extract, transform, load (ETL) engine. It will automate the cumbersome process of data preparation, transformation, and schema evolution, readying your data for analytics.
Step 3
After ETL, data is securely stored in an Amazon S3 bucket and is encrypted at rest and in transit. IAM policies will be configured for granular access control.
Step 4
API Gateway exposes the clean data stored in the Amazon S3 bucket to external customers. Amazon ElastiCache is used for caching frequently accessed data.
Step 5
Amazon Cognito handles user authentication, and Lambda functions manage access control based on user subscriptions or credits.
Step 6
A payment gateway manages transactions, and Lambda functions handle billing and invoicing based on usage and confirmed transactions.
Step 7
AWS Cost Explorer helps monitor and optimize operational costs. It provides insights into spending patterns and can help identify cost-saving opportunities.
Step 8
CloudWatch offers real-time monitoring, CloudTrail provides an audit log for governance, and AWS Config facilitates compliance with your organizational policies. Together, they form a robust framework for operational oversight.
Step 9
The end user interacts directly with the system by making API calls. These are authenticated by Amazon Cognito and may have access controlled by Lambda, based on subscriptions or credits.
Research Data Monetization on Premises
This architecture diagram shows how you can improve data monetization with customers that use either cloud providers other than AWS or on-premises servers.
Download the architecture diagram PDF
Step 1
This is the entry point into the AWS environment. Ingest data from relational databases Amazon RDS, object stores like Amazon S3, NoSQL stores like Amazon DynamoDB, data lakes through Lake Formation, or external APIs through API Gateway. Lambda functions can be used for custom ingestion logic.
Step 2
AWS Glue serves as the extract, transform, load (ETL) engine. It will automate the cumbersome process of data preparation, transformation, and schema evolution, readying your data for analytics.
Step 3
After ETL, data is securely stored in an Amazon S3 bucket and is encrypted at rest and in transit. IAM policies will be configured for granular access control.
Step 4
API Gateway exposes the clean data stored in the Amazon S3 bucket to external customers. Amazon ElastiCache is used for caching frequently accessed data.
Step 5
Amazon Cognito handles user authentication, and Lambda functions manage access control based on user subscriptions or credits.
Step 6
A payment gateway manages transactions, and Lambda functions handle billing and invoicing based on usage and confirmed transactions.
Step 7
AWS Cost Explorer helps monitor and optimize operational costs. It provides insights into spending patterns and can help identify cost-saving opportunities.
Step 8
CloudWatch offers real-time monitoring, CloudTrail provides an audit log for governance, and AWS Config facilitates compliance with your organizational policies. Together, they form a robust framework for operational oversight.
Step 9
The end user interacts directly with the system by making API calls. These are authenticated by Amazon Cognito and may have access controlled by Lambda, based on subscriptions or credits.
Well-Architected Pillars
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
Operational Excellence
Security
This Guidance uses AWS Data Exchange to make sure that all data listed on the exchange is handled securely. Additionally, API Gateway and Amazon Cognito confirm authentication and authorization for data access.
Reliability
This Guidance uses AWS services that are all fully managed and serverless, reducing your operational burden to reliably maintain a data product. For example, Lambda maintains high availability by using multiple Availability Zones.
Performance Efficiency
This Guidance uses AWS services that are fully managed and serverless, automatically scaling up and down as needed to maintain performance. This offloads much of the burden of management from small tech teams while helping nonprofits achieve strict compliance requirements.
Cost Optimization
This Guidance uses serverless AWS services, so you only pay for what you use rather than provisioning resources that incur costs while idle. And because they are fully managed, you reduce the total cost of ownership. Additionally, you can use AWS Cost Explorer to track your resource usage and costs so that you can make adjustments as needed to stay within your budget.
Sustainability
This Guidance uses serverless AWS services, which scale up and down to meet demand. This reduces energy usage because these services run more efficiently, and you don’t need to provision unnecessary resources that would consume energy while idle.