Marshalling client traffic to applications running in Kubernetes clusters, has for many years been the job of ingress controllers, using the Ingress API. However, the Ingress API has many limitations; it’s terse by design, ambiguous in places, and has no formal means for extending its limited capabilities. In short, without enhancement, it doesn’t meet the needs of the traffic management use cases in the real world. The Kubernetes community recognized the need for a better solution, and after several years in the making, this arrived in the form of the Gateway API.
The Kubernetes Gateway API reached GA status in October 2023 and continues to evolve as new features are identified and validated within the community. There are many improvements over the Ingress API:
- Role-based – multiple object types (
GatewayClass,Gateway,HTTPRoute,TCPRoute, and so on) replace the monolithic Ingress object type, to better reflect the fact that multiple actors need to control the configuration of the end-to-end ingress experience - Expansive – the Ingress API focuses on applications using the HTTP protocol, whereas the Gateway API caters for numerous protocols at different layers in the networking stack (HTTP/S, gRPC, TCP, UDP)
- Expressive – the limited traffic management features in the Ingress API have been augmented to include traffic splitting and mirroring, request and response header modification, HTTP redirects and rewrites, and much more
- Extendable – the unofficial, unregulated method of feature extension in the Ingress API, annotations, has been replaced with a more formal method of extension through ‘policy attachment‘
In this tutorial, you’ll use the Gateway API to configure ingress traffic to an example application running in a local Kubernetes cluster. You’ll be able to differentiate between the different Gateway API object types and see how they fit together to provide the full ingress experience for an application.
Prerequisites
This tutorial uses a local development cluster using the Kind tool, where the cluster nodes are Docker containers. Docker is a prerequisite for using Kind. Follow the Quick Start guide to install Kind and to provision a local Kubernetes cluster.
To try and emulate operating in a cloud environment, the Cloud Provider for Kind should also be deployed. This will enable you to establish a local load balancer for use with the Gateway API.
Here’s a complete list of the prerequisites:
- Docker (macOS, Windows, Linux – Docker Desktop or Linux – Docker Engine)
- Kind
- Cloud Provider for Kind
- Kubernetes CLI – kubectl (macOS, Windows, Linux)
The Kubernetes configuration used in the tutorial can be found in the following GitHub repo:
Deploy an Application
With a Kubernetes cluster up and running, it’s time to deploy a sample application to the cluster, which we eventually hope to consume from a web browser, once we’ve configured ingress using the Gateway API. The app simply displays a static web page providing some information about its environment.
Step 1: Apply the Kubernetes Configuration for the App
The application can be deployed to the cluster using kubectl:
kubectl apply -f \
https://raw.githubusercontent.com/nbrownuk/gateway-api-tutorial/main/nginxhello.yamlStep 2: Check the App’s Status
And, we can check that it’s started up correctly, by retrieving the Pod, Service and Deployment objects that have the label app.kubernetes.io/name=nginxhello. We should see something like this:
$ kubectl -n nginxhello get po,svc,deploy
NAME READY STATUS RESTARTS AGE
pod/nginxhello-cbfb6bbb6-zcqqq 1/1 Running 0 9s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nginxhello ClusterIP 10.96.243.155 <none> 80/TCP 9s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginxhello 1/1 1 1 9s
We’ve got a Deployment with a single replica Pod, which is fronted by a Service of type ClusterIP.
With the app up and running, it’s time to turn our attention to the configuration of the Gateway API.
Deploy a Gateway Controller
Just like the older Ingress API in Kubernetes, there is no in-tree controller to act on instance objects of the Gateway API that are applied to the cluster. Instead, we have to deploy a third-party gateway controller of our choice. If you’re using an Amazon EKS cluster, it’s likely to be the AWS Gateway API Controller. And, if you’re using Microsoft’s AKS, it’s likely to be the Application Gateway for Containers ALB Controller. But, there are a whole bunch of different cloud-native proxies that implement the Gateway API, too, including the Envoy Gateway. The Envoy Gateway is based on the Envoy proxy, and it’s what we’ll use in this tutorial as the implementation of the Gateway API.
Step 1: Install the Envoy Gateway
Unlike the Ingress API, which is a constituent part of the default set of Kubernetes APIs exposed by the API server, the Gateway API is provided as Custom Resource Definitions (CRDs). That means the CRDs need to be installed before they can be used. Sometimes, this installation is performed as part of the installation of the chosen gateway controller. Installing the Envoy Gateway gives us the CRDs, as well as the controller itself. There are a few ways to install the Envoy Gateway, but the project provides a handy install YAML file as part of its GitHub releases:
kubectl apply --server-side -f \
https://github.com/envoyproxy/gateway/releases/latest/download/install.yamlIt will take a few moments before the Envoy Gateway is ready in the cluster.
Step 2: Check the Status of the Envoy Gateway
The installation should give us the Envoy Gateway deployed to a new namespace in our cluster, envoy-gateway-system, which we can check by retrieving the Pod, Service and Deployment objects in that namespace:
$ kubectl -n envoy-gateway-system get po,svc,deploy
NAME READY STATUS RESTARTS AGE
pod/envoy-gateway-654d5d67f8-pzmr6 1/1 Running 0 34s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envoy-gateway ClusterIP 10.96.79.24 <none> 18000/TCP,18001/TCP,18002/TCP,19001/TCP 34s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envoy-gateway 1/1 1 1 34s
If the Pod status is Running, all should be well with the gateway controller.
Step 3: Create a GatewayClass
But, it’s possible to run more than one gateway controller in a cluster; different controllers for different purposes, or horses for courses. So, we must associate the gateway controller with a GatewayClass object that can be referenced by other Gateway API objects we want to create, so that the correct controller is selected for the job. The GatewayClass definition that we need for the Envoy Gateway looks like this:
# gateway-class.yaml
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: envoy-gateway
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controllerThe value of the controllerName field is the same as the default name associated with the Envoy Gateway controller. We can see this by inspecting the ConfigMap that is used to configure the Envoy Gateway during installation:
$ kubectl -n envoy-gateway-system get cm envoy-gateway-config \
-o "jsonpath={.data['envoy-gateway\.yaml']}"
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyGateway
gateway:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
logging:
level:
default: info
provider:
type: Kubernetes
Note the value of the controllerName in the EnvoyGateway definition. So, the GatewayClass object definition we’re about to use has a correlation with the Envoy Gateway by virtue of the controllerName. But, it needs to be applied to the cluster to take effect:
kubectl apply -f \
https://raw.githubusercontent.com/nbrownuk/gateway-api-tutorial/main/gateway-class.yamlStep 4: Examine the GatewayClass Object
Once the cluster-scoped object has been created in the cluster, we can check its status:
$ kubectl get gc envoy-gateway
NAME CONTROLLER ACCEPTED AGE
envoy-gateway gateway.envoyproxy.io/gatewayclass-controller True 7s
It should have an ‘Accepted’ status of ‘True’ to indicate the Envoy Gateway has accepted processing on behalf of the GatewayClass. We’re now set up to make use of the Envoy Gateway to handle ingress traffic to the cluster.
Configure a Gateway
The next part of the process is to configure another Gateway API component for our scenario; a Gateway object.
Step 1: Define a Gateway
A Gateway defines a set of ‘logical endpoints’ that are associated with an IP address. You can think of a Gateway as defining the characteristics of the traffic that can enter the cluster at the edge. Gateway controllers act on Gateway definitions to allow the ingress of traffic to the cluster. Depending on its type, the gateway controller might provision cloud infrastructure resources, or configure a proxy on behalf of the Gateway. The endpoints can be addressed by clients internal or external to the cluster, that wish to consume the apps running in the cluster. The logical endpoints are called ‘Listeners’, and are constituent parts of Gatewaydefinitions:
# gateway.yaml
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: http-gw
namespace: nginxhello
spec:
gatewayClassName: envoy-gateway
listeners:
- name: http
protocol: HTTP
port: 80Listeners can define different network protocols; HTTP, HTTPS, TCP and UDP. Here, we’ve got a Listener specifying the HTTP protocol for traffic destined for the Gateway on port 80. Notice also that the Gateway references the Envoy Gateway instance with the associated GatewayClass named envoy-gateway. It’s this gateway controller that acts on its behalf.
Step 2: Create a Gateway
The Gateway definition needs to be applied to the cluster:
kubectl apply -f \
https://raw.githubusercontent.com/nbrownuk/gateway-api-tutorial/main/gateway.yamlOnce the API server has created the object, the Envoy Gateway acts on the definition to enable the ingress of HTTP traffic on port 80. But, what actions does the Envoy Gateway take? We can check in on the envoy-gateway-system namespace to see:
$ kubectl -n envoy-gateway-system get po,svc,deploy \
-l gateway.envoyproxy.io/owning-gateway-name=http-gw
NAME READY STATUS RESTARTS AGE
pod/envoy-nginxhello-http-gw-ca0a3c04-55ddfb67fc-6njpv 2/2 Running 0 16s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envoy-nginxhello-http-gw-ca0a3c04 LoadBalancer 10.96.21.92 172.18.0.3 80:32597/TCP 16s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envoy-nginxhello-http-gw-ca0a3c04 1/1 1 1 16s
There are some additions to the namespace that have been brought about by the Envoy Gateway. Now there is a replicated deployment of the Envoy proxy, exposed via a LoadBalancer type Service, with an externally addressable IP address on port 80. The IP address, provisioned by the Cloud Provider for Kind, is 172.18.0.3. Be sure to note which EXTERNAL-IP your env has given you – you’ll need it in the next steps! Clients can send HTTP requests to this IP address when they need to communicate with an app running in the cluster. The set up looks something like this:
Step 3: Examine the Gateway Object
We can also check the status of the created Gateway object:
$ kubectl -n nginxhello get gtw http-gw
NAME CLASS ADDRESS PROGRAMMED AGE
http-gw envoy-gateway 172.18.0.3 True 42s
This tells us the IP address associated with the Gateway (it’s the same as the externally addressable LoadBalancer Service), and that it has been programmed into the data plane by the Envoy Gateway.
But, which app do client requests get sent to? We haven’t yet defined a route to a backend service running in the cluster.
Define Routes
The final piece of the jigsaw is to define an HTTPRoute object that will allow traffic to be sent to the application that we deployed right at the beginning of this tutorial.
Step 1: Define an HTTPRoute
The HTTPRoute will look like the following:
# httproute.yaml
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: nginxhello
namespace: nginxhello
spec:
parentRefs:
- name: http-gw
hostnames:
- 172-18-0-3.nip.io # to be replaced with your EXTERNAL-IP
rules:
- backendRefs:
- name: nginxhello
port: 80Firstly, there is a reference to the Gateway that will be responsible for this route, the one we’ve just established in the cluster, called http-gw. The HTTPRoute will be ‘attached’ to the HTTP Listener in the Gateway. Then, we define a list of hostnames that must match an incoming HTTP request host header. There is a single entry; 172-18-0-3.nip.io1. We’re using the nip.io service to give us a domain name for the IP address of the LoadBalancer service associated with the Gateway in our local environment. Finally, we have a single rule, which applies no conditions, and simply references the ClusterIP Service that fronts our application. Any client HTTP requests addressed to the domain name will get routed to the application as a result.
Step 2: Create an HTTPRoute
The HTTPRoute needs to be applied to the cluster.
kubectl apply -f \
https://raw.githubusercontent.com/nbrownuk/gateway-api-tutorial/main/httproute.yamlStep 3: Examine the HTTPRoute Object
The first check we should make is to ensure the HTTPRoute object was created successfully:
$ kubectl -n nginxhello get httproute nginxhello
NAME HOSTNAMES AGE
nginxhello ["172-18-0-3.nip.io"] 11s
This response doesn’t tell us too much, other than the object has been created successfully. What is of more importance to us, is whether the HTTPRoute has been accepted by the Gateway, and we need to get more information by describing the object:
kubectl -n nginxhello describe httproute nginxhelloWe’ll get a lot of information back, but if we look out for the ‘status’ information provided, we should see something like this:
<snip>
Status:
Parents:
Conditions:
Last Transition Time: 2024-05-29T13:41:39Z
Message: Route is accepted
Observed Generation: 1
Reason: Accepted
Status: True
Type: Accepted
<snip>
The status tells us that the HTTPRoute has been accepted by the Gateway, and is now attached to it for processing purposes. Our scenario is now complete:
We’re good to go!
Step 4: Consume the Application
The final step is to use a web browser to attempt to consume the application, by using the hostname specified in the HTTPRouteas the URL. It’s a non-TLS connection, so do accept the security exception, and the app should respond to the request accordingly:
Conclusion
This tutorial provides a basic introduction to the capabilities of the Gateway API. It has much more to offer and represents a big improvement on the frailties of the Ingress API for ingress scenarios. It still has a way to go before it reaches full maturity, but it’s already mature enough to be used in production clusters. If you’re considering moving your existing Ingress definitions over to the Gateway API, check out the Ingress2Gateway utility, which provides translation for a growing set of ingress providers.
If the IP address of your Gateway is different, be sure to amend the hostname accordingly. ↩
Learn more:
