Enabling data persistence in microservices

Tabby Ward and Balaji Mohan, Amazon Web Services (AWS)

December 2023 (document history)

Organizations constantly seek new processes to create growth opportunities and reduce time to market. You can increase your organization's agility and efficiency by modernizing your applications, software, and IT systems. Modernization also helps you deliver faster and better services to your customers.

Application modernization is a gateway to continuous improvement for your organization, and it begins by refactoring a monolithic application into a set of independently developed, deployed, and managed microservices. This process has the following steps:

Although you can use a monolithic application architecture for some use cases, modern application features often don't work in a monolithic architecture. For example, the entire application can't remain available while you upgrade individual components, and you can't scale individual components to resolve bottlenecks or hotspots (relatively dense regions in your application's data). Monoliths can become large, unmanageable applications, and significant effort and coordination is required among multiple teams to introduce small changes.

Legacy applications typically use a centralized monolithic database, which makes schema changes difficult, creates a technology lock-in with vertical scaling as the only way to respond to growth, and imposes a single point of failure. A monolithic database also prevents you from building the decentralized and independent components required for implementing a microservices architecture.

Previously, a typical architectural approach was to model all user requirements in one relational database that was used by the monolithic application. This approach was supported by popular relational database architecture, and application architects usually designed the relational schema at the earliest stages of the development process, built a highly normalized schema, and then sent it to the developer team. However, this meant that the database drove the data model for the application use case, instead of the other way round.

By choosing to decentralize your data stores, you promote polyglot persistence among your microservices, and identify your data storage technology based on the data access patterns and other requirements of your microservices. Each microservice has its own data store and can be independently scaled with low-impact schema changes, and data is gated through the microservice’s API. Breaking down a monolithic database is not easy, and one of the biggest challenges is structuring your data to achieve the best possible performance. Decentralized polyglot persistence also typically results in eventual data consistency, and other potential challenges that require a thorough evaluation include data synchronization during transactions, transactional integrity, data duplication, and joins and latency.

This guide is for application owners, business owners, architects, technical leads, and project managers. The guide provides the following six patterns to enable data persistence among your microservices:

The guide is part of a content series that covers the application modernization approach recommended by AWS. The series also includes:

Targeted business outcomes

Many organizations find that innovating and improving the user experience is negatively impacted by monolithic applications, databases, and technologies. Legacy applications and databases reduce your options for adopting modern technology frameworks, and constrain your competitiveness and innovation. However, when you modernize applications and their data stores, they become easier to scale and faster to develop. A decoupled data strategy improves fault tolerance and resiliency, which helps accelerate the time to market for your new application features.

You should expect the following six outcomes from promoting data persistence among your microservices: