Roberto Carratalá

Principal AI Architect - AI BU @ Red Hat. MsC in AI/ML & MsC Telco Engineer

8 minute read

How do we configure ingress routing in Service Mesh and what components are involved? And what are the main differences between OpenShift Routes and Ingress routing in Service Mesh?

Let’s Mesh in!!

This is the fourth blog post of the Service Mesh in OpenShift series. Check the earlier post:

Overview

In this blog post, we will deep dive into the traffic management and ingress routing of Service Mesh, and the components involved to get traffic into our applications deployed within our Service Mesh.

NOTE: this blog post is supported by the istio-files repository located in my personal Github

0. Prerequisites

  • OpenShift 4.x cluster (tested in a 4.3+ cluster)
  • OpenShift Service Mesh Operators installed (v1.1.1 in these blog posts)
  • Service Mesh Control Plane deployed
  • Four microservices deployed (follow the second blog post)
  • Microservices added within Service Mesh (follow the third blog post)

1. Adapt Existing K8S Services

In OpenShift, when oc new-app is used, several Kubernetes resources are created by this command. One of these resources is the Service.

Check out the service of the customer microservice deployed:

 oc get svc/customer -n $OCP_NS -o json | jq -r '[.spec.selector]'
[
  {
    "app": "customer",
    "deploymentconfig": "customer",
    "version": "v1"
  }
]

As we know, Kubernetes services match pods based on the selector we have defined. In the previous example, the app, deploymentconfig, and version labels define the match with these pods.

So, this means that only pods matching those labels will be using the loadbalancing of this specific service (be ‘behind’ the service).

But in Service Mesh, we need to define the Service with a slightly wider selector, because we will differentiate the specific versions with the Destination Rules that we will describe in the next sections.

An example of a new selector is the following:

cat istio-files/customer/kubernetes/Service.yml | grep selector -A5
 selector:
   app: customer

This selector will match any pod labelled with app: customer, and this possibility includes several versions of the same application.

Run the specific commands to delete the existing oc new-app services and apply the service adapted to istio:

$ oc delete svc/customer -n $OCP_NS
service "customer" deleted

$ oc delete svc/preference -n $OCP_NS
service "preference" deleted

$ oc delete svc/recommendation -n $OCP_NS
service "recommendation" deleted

$ oc delete svc/partner -n $OCP_NS
service "partner" deleted

$ oc apply -f istio-files/customer/kubernetes/Service.yml -n $OCP_NS
service/customer created

$ oc apply -f istio-files/preference/kub
ernetes/Service.yml -n $OCP_NS service/preference created

$ oc apply -f istio-files/recommendation/kubernetes/Service.yml -n $OCP_NS
service/recommendation created

$ oc apply -f istio-files/partner/kubernetes/Service.yml -n $OCP_NS
service/partner created

2. Enabling Ingress Routing

Now that we have the proper Services patches, we need to enable the ingress routing in order to reach and consume our microservices outside of the Mesh and the OpenShift cluster.

2.1 OpenShift Routes vs Ingress Service Mesh

There is a main difference between the ingress of OpenShift Routes and the Ingress Routing with Service Mesh.

The OpenShift Route uses the OpenShift Ingresscontroller/Router (Haproxy) for getting the traffic into the cluster, the router points to the specific service with the labels, and through that service reaches the specific pods of our applications:

  • OpenShift Route Ingress by Default:
Route (HAProxy/Router) => SVC1 (ip-pod-1, ip-pod-2, ...)

So far so good, right? But within the Service Mesh, using this Route will miss the features and capabilities of Service Mesh in those services which are externally exposed. This is because some policies and rules are applied on the client proxy.

So in order to take advantage of the capabilities of Istio in our ingress routing, we will use the following Istio resources:

  • Ingress Gateway
  • VirtualService
  • DestinationRule

We’ll explain these components in a bit, but for now, at a high level the ingress routing within Mesh will be the following:

  • Ingress Routing within Service Mesh
Route (HAProxy/Router) => istio-ingressgateway  => SVC1 (ip-pod-1, ip-pod-2, ...)

So for getting the traffic into the cluster and into the Service Mesh, an ingress gateway will be used, but instead of accessing through the same ingress gateway (and using several paths as /customer or /partner) we’re using a Route that points to the istio ingress gateway.

After the definition of the ingress gateway and using the VirtualService, the requests will be routed through this ingress gateway to the services of k8s and therefore will be routed finally to our Pods.

It may not seem easy, but in fact it is more straightforward than it looks. Let’s take a look at the components used in this process.

2.2 Components of Service Mesh for Traffic Management

  • A virtual service lets you configure how requests are routed to a service within an Istio service mesh, building on the basic connectivity and discovery provided by Istio and your platform.

  • Example1 of VirtualService using a gateway (customer-gw in this case):

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: customer
  namespace: ${NAMESPACE}
spec:
  gateways:
  - customer-gw
  hosts:
  - customer-${NAMESPACE}-istio-system.${APP_SUBDOMAIN}
  http:
  - route:
    - destination:
        host: customer
        subset: version-v1
      weight: 100
  • Example2 of VirtualService using User headers to segregate traffic:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        end-user:
          exact: jason
    route:
    - destination:
        host: reviews
        subset: v2
  - route:
    - destination:
        host: reviews
        subset: v3
  • Destination Rules are applied after virtual service routing rules are evaluated, so they apply to the traffic’s “real” destination.

You use destination rules to specify named service subsets, such as grouping all a given service’s instances by version. You can then use these service subsets in the routing rules of virtual services to control the traffic to different instances of your services.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: my-destination-rule
spec:
  host: my-svc
  trafficPolicy:
    loadBalancer:
      simple: RANDOM
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2
    trafficPolicy:
      loadBalancer:
        simple: ROUND_ROBIN

In fact, think of virtual services as how you route your traffic to a given destination, and then you use destination rules to configure what happens to traffic for that destination.

  • Gateway: You use Gateways to manage inbound and outbound traffic for your mesh, letting you specify which traffic you want to enter or leave the mesh.
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: ext-host-gwy
spec:
  selector:
    app: my-gateway-controller
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
    - ext-host.example.com
    tls:
      mode: SIMPLE
      serverCertificate: /tmp/tls.crt
      privateKey: /tmp/tls.key

2.3 Creating service mesh objects for our microservices

To expose our microservices with an ingress routing, we need to create several objects, some of them reviewed in the previous section.

Be aware that after applying these Istio objects, the old routes will no longer work, and the new routes will no longer be in the namespace of our microservices; every route and ingress gateway will be located in the istio-system namespace.

So in our case the objects that will be applied will be the following:

Route (HAProxy/Router) => istio-ingressgateway  => SVC1 (ip-pod-1, ip-pod-2, ...)
  • Route (ns => istio-system)
  • Gateway (ns => istio-tutorial)
  • VirtualService (ns => istio-tutorial)
  • DestinationRule (ns => istio-tutorial) - Enable MTLS

Render and apply the objects to enable the ingress routing:

$ export APP_SUBDOMAIN=$(oc get route -n istio-system | grep -i kiali | awk '{ print $2 }' | cut -f 2- -d '.')
$ echo $APP_SUBDOMAIN
apps.ocp4.rglab.com
export OCP_NS="istio-tutorial"

$ cat istio-files/customer-ingress_mtls.yml | NAMESPACE=$(echo $OCP_NS) envsubst | oc apply -f -
route.route.openshift.io/customer created
gateway.networking.istio.io/customer-gw created
virtualservice.networking.istio.io/customer created
destinationrule.networking.istio.io/customer created

2.4 Let’s dig into the ingress routing objects:

First the route:

$ oc get route -n istio-system customer -o yaml --export
...
spec:
  host: customer-istio-tutorial-istio-system.apps.ocp4.rglab.com
  port:
    targetPort: http2
  to:
    kind: Service
    name: istio-ingressgateway
    weight: 100
  wildcardPolicy: None
status:
  ingress: null

As we can see, the route points to the service of the istio-ingressgateway using the host of the {app}{app-ns}{istio-ns}.apps and the port for http2.

For the Gateway, the customer-gw is defined in the namespace of our apps:

$ oc get gateway -n istio-tutorial customer-gw -o yaml | yq .spec
{
  "selector": {
    "istio": "ingressgateway"
  },
  "servers": [
    {
      "hosts": [
        "customer-istio-tutorial-istio-system.apps.ocp4.rglab.com"
      ],
      "port": {
        "name": "http2",
        "number": 80,
        "protocol": "HTTP2"
      }
    }
  ]
}

As we can see, the selector has istio: ingressgateway, defining that it is linked with the Istio ingressgateway, and has the host and the port defined also in the route.

$ oc get virtualservice -n istio-tutorial customer -o yaml | yq .spec
{
  "gateways": [
    "customer-gw"
  ],
  "hosts": [
    "customer-istio-tutorial-istio-system.apps.ocp4.rglab.com"
  ],
  "http": [
      "route": [
        {
          "destination": {
            "host": "customer",
            "subset": "version-v1"
          }
        }
      ]
    }
  },
}

The VirtualService points to the gateway of customer-gw, and sets the route request to the customer host with the subset of version-v1.

Finally the DestinationRule is defined as:

$ oc get destinationrule -n istio-tutorial customer -o yaml | yq .spec
{
  "host": "customer",
  "subsets": [
    {
      "labels": {
        "version": "v1"
      },
      "name": "version-v1"
    },
  "trafficPolicy": {
    "tls": {
      "mode": "ISTIO_MUTUAL"
    }
  }
}

The destination rule sets the subset of versions (currently version-v1, but in the next labs we will be expanding them) that we have in place. It also defines the host that belongs to this DestinationRule, and an important feature: enabling Mutual TLS.

Another blog post will be dedicated to analyzing Mutual TLS in depth, but for now note that with this traffic policy, MTLS is enabled.

3. Testing the ingress routing of our microservices in Service Mesh

Check the routes generated in the istio-system namespace:

$ oc get route -n istio-system customer
NAME       HOST/PORT                                                  PATH   SERVICES               PORT    TERMINATION   WILDCARD
customer   customer-istio-tutorial-istio-system.apps.ocp4.rglab.com          istio-ingressgateway   http2                 None

Curl them to test that everything is OK:

$ curl customer-istio-tutorial-istio-system.apps.ocp4.rglab.com -I
HTTP/1.1 200 OK
content-type: text/plain;charset=UTF-8
content-length: 0

It works!

Check out the traffic flow with Kiali:

IMPORTANT: Old routes do not work anymore; use the new routes under istio-system:

$ oc get route -n istio-tutorial customer
NAME       HOST/PORT                                     PATH   SERVICES   PORT       TERMINATION   WILDCARD
customer   customer-istio-tutorial.apps.ocp4.rglab.com          customer   8080-tcp                 None

$ curl -I customer-istio-tutorial.apps.ocp4.rglab.com
HTTP/1.0 503 Service Unavailable

4. Apply the istio routing to the partner microservice

Now that we’re aware that everything is working with customer microservice ingress routing, let’s apply this to the partner microservice:

$ cat istio-files/partner-ingress_mtls.yml | NAMESPACE=$(echo $OCP_NS) envsubst | oc apply -f -
route.route.openshift.io/partner created
gateway.networking.istio.io/partner-gw created
virtualservice.networking.istio.io/partner created
destinationrule.networking.istio.io/partner created

Test it to check if everything is OK:

$ curl -I partner-istio-tutorial-istio-system.apps.ocp4.rglab.com
HTTP/1.1 200 OK
x-application-context: application
content-type: text/plain;charset=UTF-8
content-length: 76

$ curl partner-istio-tutorial-istio-system.apps.ocp4.rglab.com
partner => preference => recommendation v1 from 'recommendation-2-kjpsc': 9

So now, we have in place the two routes for our ingress routing system:

$ oc get routes -n istio-system | egrep "customer|partner"
customer               customer-istio-tutorial-istio-system.apps.ocp4.rglab.com          istio-ingressgateway   http2                        None
partner                partner-istio-tutorial-istio-system.apps.ocp4.rglab.com           istio-ingressgateway   http2                        None

Check out the part five of this blog series in Blue Green deployments in Service Mesh

NOTE: Opinions expressed in this blog are my own and do not necessarily reflect that of the company I work for.

Happy Service Meshing!

You may also enjoy