North-South Entry Gateway Selection
A write-up of our north-south gateway selection for a hybrid cloud spanning China and overseas, comparing the pros and cons of Istio, Envoy and other gateway options.
The road to selecting a north-south gateway
Background
Our container cloud is a hybrid spanning China and overseas. By provider it includes:
- Alibaba Cloud
- Huawei Cloud
- AWS
- Oracle
- Self-hosted
By region it includes:
- Mainland China
- Overseas
- Frankfurt
- Russia
- Singapore
- ……
So we wanted a gateway solution that
- covers every cluster and every region
- is preferably a managed cloud product
- uses one uniform integration approach across clusters
- covers
- the capabilities of Ingress Nginx
- business developers’ everyday gateway needs
Phase one targets the north-south gateway; phase two will consider a service mesh (east-west) depending on how things go.
So this write-up focuses on north-south gateway selection, but whatever we pick must also be able to support a service mesh later.
Given that background, the options available in the industry look roughly like this:
First, the basic capabilities of a business gateway:
- HTTP routing
- HTTP traffic splitting
- Canary traffic rollout
- Blue-green traffic rollout
- ……
- Timeout
- Rewrite
- Redirect
- Header Modify
- Request Mirror
- WebSocket
- Retry
- CORS
- Rate Limits
- Circuit Breaking
These capabilities also cover everything Ingress Nginx does.
For the gateway part, you might wonder why we did not evaluate Alibaba Cloud’s MSE.
Mainly because MSE can only be deployed in Alibaba Cloud ACK — it cannot manage non-Alibaba Cloud ACK clusters or self-hosted clusters.
Now let’s go through the pros and cons of each option.
Pros and cons of each gateway
Huawei Cloud ASM
Built on Istio Gateway; supports a service mesh, but only via the sidecar mode.
Advantages
- ASM is fully compatible with Istio CRDs
Disadvantages
- The ASM base edition supports a single cluster, a maximum of 200 instances, and no SLA guarantee
- UCS service mesh is restricted for commercial use (nobody has used it yet)
- Note 1:
the ASM Enterprise edition has been taken offline
- Note 1:
- Cannot manage clusters from other clouds or self-hosted clusters
- Also has the route priority problem
Alibaba Cloud ASM
Built on Istio Gateway; supports a service mesh in both sidecar and ambient modes, though ambient is not recommended for production at this stage.
Advantages
- Provides an SLA guarantee
- No need to operate the components yourself
- Supports managing clusters from other clouds and self-hosted clusters
- For business clusters where Alibaba Cloud has no region nearby, or where overseas network quality is poor, there is a targeted solution: ASM remote control plane
- Full feature set
- One control plane can manage multiple Kubernetes clusters
- Note: these clusters are essentially mirror clusters — the ASM CRDs deployed in-cluster and the routes pushed down are all identical, so this feature is not much use
Disadvantages
- Because it is directly built on Istio Gateway, routing order is a problem
- Currently supports Kubernetes Gateway API v1.1.0, and not the v1.2.0 features
- Standards
- HTTPRoute timeouts
- Backend protocol support for WebSocket and HTTP/2
- Standards
Istio Gateway / Kubernetes Gateway API
For Istio Gateway there are two options:
- Going with the standard — that is, the Kubernetes Gateway API
- Standardization: the Kubernetes Gateway API aims to be a standard for describing how services are exposed in a Kubernetes cluster. That means it is not limited to working with one particular service mesh or ingress controller (such as Istio). By adopting the Gateway API, you can switch implementations or providers more easily without rewriting your configuration.
- Conceptual consistency: consistent with the MSE and APISIX we had used before
- Better integration: because the Gateway API is a Kubernetes-native API, it integrates better with other Kubernetes-native tools and systems
- Route priority support
- Talking directly to Istio Gateway
- Today Istio Gateway covers more than the Kubernetes Gateway API — for example circuit breaking and rate limiting
- But it introduces Istio Gateway concepts such as VirtualService and DestinationRule, raising the learning curve for users
- Routing order is non-deterministic
At this point, looking at route priority alone eliminates the cloud ASM options and talking directly to Istio Gateway.
But a few more things to note:
- The Kubernetes Gateway API currently does not support circuit breaking or rate limiting
- For cross-origin support we also have to wait for Milestone v1.3.0 (current version v1.2.1; the next should be v1.3.0, but the exact release date is uncertain)
Service Mesh
From the official site, Istio’s support for the Kubernetes Gateway API as a service mesh has already gone GA.
Route priority in the Kubernetes Gateway API
Route configurations for proxies or load balancers generated from HTTPRoutes must prioritize matches according to the following criteria, continuing to the next criterion on ties. Among all rules specified on applicable routes, priority is given to matches with:
- An “exact” path match.
- A “prefix” path match with the most characters.
- A method match.
- The greatest number of header matches.
- The greatest number of query parameter matches.
Note: the priority of regular expression path matches is implementation-specific.
If a tie still exists between routes, match precedence must be determined by the following criteria in order, continuing to the next criterion on ties:
- The earliest route based on creation timestamp.
- The first route in “{namespace}/{name}” alphabetical order.
If a tie still exists within an HTTPRoute, match precedence goes to the first matching rule (in list order) that satisfies the above.
If a rule matching the request fails to attach to the request’s parent, HTTP 404 must be returned.
Source:
https://gateway-api.sigs.k8s.io/reference/spec/#gateway.networking.k8s.io/v1.HTTPRouteRule
Judging by where the note sits in the doc, multiple HTTPRouteMatch entries within a single HTTPRouteRule should satisfy these rules. What about across multiple HTTPRouteRules?
Let’s verify both cases.
Verification results
We verified the common cases:
- Priority among multiple prefix matches
- Priority between a prefix match and an exact match
As the results show, Istio routing follows the Kubernetes Gateway API spec.
Result
From the options listed here, the most viable paths today are:
- Deploy Istio Gateway ourselves and talk directly to the Kubernetes Gateway API CRDs.
- In practice this still waits for Milestone v1.3.0 to ship and Istio to support it
- Or wait for Alibaba Cloud ASM to support Milestone v1.3.0
Misc
Kubernetes version range supported by the cloud ASM offerings, and their Istio versions
Sidecar vs Ambient
https://istio.io/latest/zh/docs/overview/dataplane-modes/#choosing-between-sidecar-and-ambient
Istio and the Kubernetes Gateway API
How does Istio implement Kubernetes Gateway API route priority?
This article is the English edition of a post originally published in Chinese.