2 Challenges when Adopting a Kubernetes API Gateway

Building applications using the microservices pattern and deploying these services onto Kubernetes has become the de facto approach for running cloud-native applications today. In a microservice architecture, a single application is decomposed into multiple microservices. Each microservice is owned by a small team that is empowered and responsible to make the right decisions for the specific microservice.

This responsibility typically extends from the edge of the system where the user requests arrive, right the way through to the service’s business logic and down into the associated messaging and data store schema.

When integrating an API gateway with a microservices-based application running on Kubernetes, you must consider two primary challenges:

• How to scale the management of 100s of services and the associated APIs; and
• How the gateway can support a broad range of microservice architectures, protocols, and configuration that typically spans the entire Edge Stack.

The API Gateway: A Focal Point with Microservices

An API gateway is at the core of how APIs are managed, secured, and presented. It is deployed as a software component (or series of components) on virtual machines or within Kubernetes, and acts as the single entry point into a system. The primary responsibility of an API gateway is to enable multiple APIs, microservices, and backend systems to be accessed reliably and safely by users.

Microservices and Kubernetes provide implementation flexibility. For example, one team may elect to expose a container-based microservice at the edge of the system (the boundary between the internal services and end users) as a set of REST APIs over HTTP. Another team may choose Protobufs and gRPC. A team with real-time streaming requirements may expose their microservice over WebSocket APIs. Any API gateway deployed within Kubernetes must support all of these protocols.

Each team is not only free to make these choices, but they are also responsible for the consequences; this often translates into “you build it, you run it”. Although not every organization subscribes completely to this way of working, every microservice team needs to be able to understand, diagnose, and configure all aspects of the handling of each service and each user’s request into the application. The diversity of runtime requirements related to applications and APIs means that each team will be working with all layers within the edge stack, for example, the dynamic request handling, the WAF, and any caching implementation.

The development paradigm of microservices -- independent, empowered, and responsible teams -- creates a new set of challenges for microservice teams working with API Gateways, Kubernetes ingress and the edge.

In this article, we identify two important challenges for the edge: managing independent microservices, and having access to a comprehensive edge stack.

Challenge #1: Scaling Edge Management

The challenges of managing the edge increase with the number of microservices deployed

In a microservices architecture, engineers will be managing many more services and applications. Each team needs to be able to manage their services independently, in order for releases to be decoupled from other teams schedules. The traditional approach to exposing applications at the edge is typically done through a centralized operations or platform team. However, a single ops team cannot scale to handle the volume of changes that are necessary when an organization has hundreds of microservices.

Typical changes that require modification of configuration at the edge:

• New version of a service being deployed.
• Modifying endpoints, routing instructions, or the associated backend services.
• Changes to authentication and authorization services.
• Modification of nonfunctional requirements, such as rate limiting, timeouts, retry patterns, and circuit-breaking.
• User testing of new functionality, for example, enabling a feature for a small subset of beta test users.

Adopting a microservices-based architecture will result in a significant rise in the number of releases. This increase only magnifies the edge management challenges and increases the strain on a centralized approach to operation.

Challenge #2: Supporting a Diverse Range of Edge Requirements

Microservices introduce a number of new concerns at the edge

The microservice architecture enables architectural flexibility. Application developers take advantage of this flexibility to choose the programming language and architecture that best fits the specific requirements of the service. The edge needs to support the broad spectrum of functionality that need to be exposed to users, regardless of architecture. This extends the traditional role of the API gateway, and some of the challenges related to the need of consolidated tooling at the edge includes:

• The ability to adeptly route a wide variety of protocols. Common protocols include HTTP/1.1, HTTP/2, WebSockets, gRPC, gRPC-Web, and TCP.
• Provide the full aggregate set of edge capabilities needed by any specific service, ranging from traffic management to observability to authentication and beyond.
• Exposing these capabilities in a self-service model for application developers.

Encouraging a diversity of implementation within microservice teams allows engineers to choose the “right tool for the job”. However, consolidation of the underlying platform offers many benefits. Rather than allowing developers to build bespoke implementations for additional protocol support or security handling, it is much more manageable and scalable to present them with a pre-approved “buffet” of options at the edge, so that they can pick and choose the most appropriate combination of functionality.

Summary

As organizations adopt Kubernetes and shift to a microservices-based architecture, a new set of challenges emerge at the boundary between end users and the internal microservices. This “edge” of the system, and related technologies like an API gateway, are therefore a focal point when adopting microservices. These new challenges at the edge are driven by the organizational model of microservices, where independent teams are empowered and responsible to make the right architectural and implementation decisions for a microservice.

Managing the edge of the system has always been complicated. Adding more services with a diversity of architectures only increases the demands on the edge. Platform teams must design, choose, and implement their API gateway and edge tooling accordingly.