Smart endpoints and dumb pipes - Running Containerized Microservices on AWS

Smart endpoints and dumb pipes

As your engineering organization transitions from building monolithic architectures to building microservices architectures, it will need to understand how to enable communications between microservices. In a monolith, the various components are all in the same process. In a microservices environment, components are separated by hard boundaries. At scale, a microservices environment will often have the various components distributed across a cluster of servers so that they are not even necessarily collocated on the same server.

This means there are two primary forms of communication between services:

  • Request/Response – One service explicitly invokes another service by making a request to either store data in it or retrieve data from it. For example, when a new user creates an account, the user service makes a request to the billing service to pass off the billing address from the user’s profile so that that billing service can store it.

  • Publish/Subscribe – Event-based architecture where one service implicitly invokes another service that was watching for an event. For example, when a new user creates an account, the user service publishes this new user signup event and the email service that was watching for it is triggered to email the user asking them to verify their email.

One architectural pitfall that generally leads to issues later on is attempting to solve communication requirements by building your own complex enterprise service bus for routing messages between microservices. It is much better to use a message broker such as Amazon Managed Streaming for Apache Kafka (Amazon MSK), Amazon MQ, or a decentralized integration service such as Amazon EventBridge. Amazon Simple Notification Service (Amazon SNS) can be very useful to use when there is a need to notify between application and application or between application and person, and Amazon Simple Queue Service (Amazon SQS) is a great way to queue messages for microservices and distributed systems. Microservices architectures favor these tools because they enable a decentralized approach in which the endpoints that produce and consume messages are smart, but the pipe between the endpoints is dumb. In other words, concentrate the logic in the containers and refrain from leveraging (and coupling to) sophisticated buses and messaging services.

Network communication often plays a central role in distributed systems. Service meshes strive to address this issue. Here you can leverage the idea of externalizing selected functionalities. Service meshes work on a sidecar pattern where you add containers to extend the behavior of existing containers. Sidecar is a microservices design pattern where a companion service runs next to your primary microservice, augmenting its abilities or intercepting resources it is utilizing. Envoy, a sidecar container, is used with AWS App Mesh as a proxy for all ingress and egress traffic to the primary microservice. Using this sidecar pattern with Envoy you can create the backbone of the service mesh, without impacting our applications, a service mesh is comprised of a control plane and a data plane. In current implementations of service meshes, the data plane is made up of proxies sitting next to your applications or services, intercepting any network traffic that is under the management of the proxies. Envoy can be used as a communication bus for all traffic internal to a service-oriented architecture (SOA).

Ambient Mesh, a new Istio data plane mode, is designed to simplify operations, application compatibility, and help reduce the infrastructure cost. Ambient Mesh provides an option to maintain Istio’s core features of zero-trust security, telemetry, and traffic management without the complexities of sidecar.

Amazon ECS Service Connect is another new addition to simplify service-to-service communication, service discovery, and traffic observability for Amazon ECS. Service Connect helps you build applications faster by letting you focus on the application code and not on your networking infrastructure.

Managing sidecar-based service meshes can be engaging and requires prior VPC networking experience; Amazon VPC Lattice simplifies this and provides a consistent way to connect, secure, and monitor communication between your services. You can define policies for network traffic management, access, and monitoring to connect compute services in a simplified and consistent way across instances, containers, and serverless applications. VPC Lattice manages all of your service-to-service connectivity, security, and monitoring needs so that you can focus on your application logic and deliver applications faster.

Sidecars can also be used to build monitoring solutions. When you are running microservices using Kubernetes, there are multiple observability strategies, one of them is using sidecars. Due to the modular nature of the sidecars, you can use it for your logging and monitoring needs. For example, you can setup FluentBit or Firelens for Amazon ECS to send logs from containers to Amazon CloudWatch Logs. AWS Distro for Open Telemetry can also be used for gathering metrics and sending metrics off to other services. Recently AWS has launched managed Prometheus and Grafana for the monitoring/ visualization use cases.

The core benefit of building smart endpoints and dumb pipes is the ability to decentralize the architecture, particularly when it comes to how endpoints are maintained, updated, and extended. One goal of microservices is to enable parallel work on different edges of the architecture that will not conflict with each other. Building dumb pipes enables each microservice to encapsulate its own logic for formatting its outgoing responses or supplementing its incoming requests.

The following are the key factors from the twelve-factor app pattern methodology that play a role in building smart endpoints and dumb pipes:

  • Port Binding – Services bind to a port to watch for incoming requests and send requests to the port of another service. The pipe in between is just a dumb network protocol such as HTTP.

  • Backing services – Dumb pipes allow a background microservice to be attached to another microservice in the same way that you attach a database.

  • Concurrency – A properly designed communication pipeline between microservices allows multiple microservices to work concurrently. For example, several observer microservices may respond and begin work in parallel in response to a single event produced by another microservice.