Introduction

Global hybrid multi cloud applications (GHMAs) auto scale vertically and horizontally in response to spikes in request traffic and processing load.

Auto scaling mechanisms for GHMAs are available on prem, in the public cloud, or in any combination globally.

This works because GHMAs abstract the underlying deployment infrastructure and the event-driven architecture (EDA) messaging system, regardless of where they are running.

Simplified example

Let’s use a version of the simplified international payment processing flow from a previous blog post to illustrate how auto scale works for GHMAs.

Organizations typically submit a stream of payment requests to the microservices flow for processing (a real flow has more steps).

The Submit/Reply microservice accepts the submissions, sends them to the Validate service, and also replies to the submitter at the end of the flow to confirm the payment (or notifies the submitter of any processing error).

The other microservices validate the payment request message, check the submitter’s credit, check for fraud, execute the payment, and confirm the payment. The microservices in this example are deployed in Kubernetes clusters.

In this simplified version of the example, let’s assume the Validate service is running in the data center and the Fraud Check service is running on AWS.

Let’s assume that someone unexpectedly submits a large volume of payment requests all at once, perhaps for paying contract car service drivers or for paying employees through a payroll.

Let’s also assume that the large volume of payment requests are validated against some additional new security requirements and contain an unusually high number of fraud flags.

Both of these things put additional, unexpected processing loads on the flow, triggering the auto scaling mechanisms.

Options to Auto Scale a Microservice

Auto scaling mechanisms respond to an unexpected spike in request traffic or processing by increasing resources and replicating microservices.

Kubernetes clusters dynamically increase CPU, memory and other resources when configured to do so. They also spin up new pods dynamically when demand increases and spin them down automatically when demand falls off.

Auto scaling mechanisms also replicate microservices horizontally and/or vertically when needed to handle increased processing volumes, as follows:

• Horizontal auto scaling: Automatic replication of a microservice on multiple Kubernetes clusters across multiple VMs
• Vertical auto scaling: Automatic replication of a microservice within the same Kuberbetes cluster on the same VM

For example, when CPU levels are saturated across all the nodes supporting a Kubernetes cluster, the auto scaling mechanism replicates one or more microservices to a Kubernetes cluster running on another node (or nodes), for example to access additional CPU resources. This is called horizontal scaling.

For another example, memory capacity may reach its limit on one node in the Kubernetes cluster, triggering the deployment of additional microservice replicas within the cluster to access additional memory. This is called vertical scaling.

The following sections look at horizontal and vertical auto scaling in more detail.

Horizontal Scaling

Let’s look first at horizontal autoscaling. In our hypothetical example, a large volume of payment submissions and new validation requirements increases the load on the Validate service (running on prem).

The horizontal auto scaling mechanism replicates the Validate microservice across multiple Kubernetes clusters running on multiple VMs in the data center. In this example, a CPU limit configuration setting triggers the auto scaling mechanism.

The GHMA platform provisions the VMs and replicates the microservices onto the additional Kubernetes clusters and balances the incoming payment request load across them.

Vertical Scaling

Next, let’s take a look at vertical scaling. In this case the Kubernetes cluster increases the number of pods in the cluster and deploys additional microservices to the new pods.

As a larger than usual number of fraud flags stresses the AWS based Fraud Service beyond its usual capacity, a memory threshold configuration setting triggers the auto scaling mechanism to deploy additional replicas of the Fraud Check microservice within the Kuberbetes cluster. This spreads the available memory allocation across more pods to increase resources needed for processing the fraud checks efficiently.

In this scenario, Kubernetes creates new pods in the cluster, allocates additional memory resources to them, and replicates the microservices to the new pods. The GHMA platform automatically balances the event traffic across the new replicas.

Summary

Both the Validation and Fraud Check microservices take advantage of auto scaling mechanisms to meet the unexpected increase in processing, triggered by predefined CPU and memory threshold settings.

The Validate service is memory intensive and the best auto scaling technique is to replicate the microservices to additional VMs with large memory capacities.

The Fraud Check service is CPU intensive, so the best auto scaling technique is to replicate the microservice to additional pods within the cluster to reduce CPU impact.

These auto scaling techniques keep the bank’s systems on track to meet the SLA for processing the payments, despite the unexpected spike in submitted volume and processing load.

Inherent GHMA infrastructure and EDA abstractions support vertical and horizontal auto scaling for any combination of data center and public cloud infrastructure.

The Intellyx Take

Auto scaling mechanisms deliver significant value in handling uneven workloads, which are very common.

Without auto-scaling, an organization has to allocate static capacity sufficient for the largest anticipated workload. Such static capacity often sits idle and is expensive to maintain.

Adopting the GHMA approach to EDA driven microservices lays the foundation for vertical and horizontal auto scaling mechanisms independently of where those microservices are deployed – in the data center, in the cloud, or in any combination.

The Fiorano Cloud Native Integration platform abstractions support both vertical and horizontal auto scaling for any deployment topology. Configuring and using the FIorano platform for auto scaling gives you all the advantages of EDA for microservices, including Kubernetes-based deployments.

Because the connections among microservices are stateless, it’s a straightforward matter to replicate the microservices to quickly respond to spikes in load. The client doesn’t need to know which of any number of identical microservices may pick up and process the message, or where they are running.

Copyright © Intellyx BV. Intellyx is solely responsible for the content of this article. As of the time of writing, Fiorano is an Intellyx customer. No AI chatbots were used to write this content.