Deprecated
Use of
Federation v1
is strongly discouraged.Federation V1
never achieved GA status and is no longer under active development. Documentation is for historical purposes only.For more information, see the intended replacement, Kubernetes Federation v2.
This page explains how to use Kubernetes Federated Ingress to deploy a common HTTP(S) virtual IP load balancer across a federated service running in multiple Kubernetes clusters. As of v1.4, clusters hosted in Google Cloud (both Google Kubernetes Engine and GCE, or both) are supported. This makes it easy to deploy a service that reliably serves HTTP(S) traffic originating from web clients around the globe on a single, static IP address. Low network latency, high fault tolerance and easy administration are ensured through intelligent request routing and automatic replica relocation (using Federated ReplicaSets. Clients are automatically routed, via the shortest network path, to the cluster closest to them with available capacity (despite the fact that all clients use exactly the same static IP address). The load balancer automatically checks the health of the pods comprising the service, and avoids sending requests to unresponsive or slow pods (or entire unresponsive clusters).
Federated Ingress is released as an alpha feature, and supports Google Cloud Platform (Google Kubernetes Engine, GCE and hybrid scenarios involving both) in Kubernetes v1.4. Work is under way to support other cloud providers such as AWS, and other hybrid cloud scenarios (e.g. services spanning private on-premises as well as public cloud Kubernetes clusters).
You create Federated Ingresses in much that same way as traditional Kubernetes Ingresses: by making an API call which specifies the desired properties of your logical ingress point. In the case of Federated Ingress, this API call is directed to the Federation API endpoint, rather than a Kubernetes cluster API endpoint. The API for Federated Ingress is 100% compatible with the API for traditional Kubernetes Services.
Once created, the Federated Ingress automatically:
Note that in the case of Google Cloud, the logical L7 load balancer is not a single physical device (which would present both a single point of failure, and a single global network routing choke point), but rather a truly global, highly available load balancing managed service, globally reachable via a single, static IP address.
Clients inside your federated Kubernetes clusters (Pods) will be automatically routed to the cluster-local shard of the Federated Service backing the Ingress in their cluster if it exists and is healthy, or the closest healthy shard in a different cluster if it does not. Note that this involves a network trip to the HTTP(s) load balancer, which resides outside your local Kubernetes cluster but inside the same GCP region.
This document assumes that you have a running Kubernetes Cluster Federation installation. If not, then see the federation admin guide to learn how to bring up a cluster federation (or have your cluster administrator do this for you). Other tutorials, for example this one by Kelsey Hightower, are also available to help you.
You must also have a basic working knowledge of Kubernetes in general, and Ingress in particular.
You can create a federated ingress in any of the usual ways, for example, using kubectl:
kubectl --context=federation-cluster create -f myingress.yaml
For example ingress YAML configurations, see the Ingress User Guide.
The --context=federation-cluster
flag tells kubectl to submit the
request to the Federation API endpoint, with the appropriate
credentials. If you have not yet configured such a context, see the
federation admin guide or one of the
administration tutorials
to find out how to do so.
The Federated Ingress automatically creates and maintains matching Kubernetes ingresses in all of the clusters underlying your federation. These cluster-specific ingresses (and their associated ingress controllers) configure and manage the load balancing and health checking infrastructure that ensures that traffic is load balanced to each cluster appropriately.
You can verify this by checking in each of the underlying clusters. For example:
kubectl --context=gce-asia-east1a get ingress myingress
NAME HOSTS ADDRESS PORTS AGE
myingress * 130.211.5.194 80, 443 1m
The above assumes that you have a context named ‘gce-asia-east1a’ configured in your client for your cluster in that zone. The name and namespace of the underlying ingress automatically matches those of the Federated Ingress that you created above (and if you happen to have had ingresses of the same name and namespace already existing in any of those clusters, they will be automatically adopted by the Federation and updated to conform with the specification of your Federated Ingress. Either way, the end result will be the same).
The status of your Federated Ingress automatically reflects the real-time status of the underlying Kubernetes ingresses. For example:
kubectl --context=federation-cluster describe ingress myingress
Name: myingress
Namespace: default
Address: 130.211.5.194
TLS:
tls-secret terminates
Rules:
Host Path Backends
---- ---- --------
* * echoheaders-https:80 (10.152.1.3:8080,10.152.2.4:8080)
Annotations:
https-target-proxy: k8s-tps-default-myingress--ff1107f83ed600c0
target-proxy: k8s-tp-default-myingress--ff1107f83ed600c0
url-map: k8s-um-default-myingress--ff1107f83ed600c0
backends: {"k8s-be-30301--ff1107f83ed600c0":"Unknown"}
forwarding-rule: k8s-fw-default-myingress--ff1107f83ed600c0
https-forwarding-rule: k8s-fws-default-myingress--ff1107f83ed600c0
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
3m 3m 1 {loadbalancer-controller } Normal ADD default/myingress
2m 2m 1 {loadbalancer-controller } Normal CREATE ip: 130.211.5.194
Note that:
To render the underlying ingress shards healthy, you need to add backend Pods behind the service upon which the Ingress is based. There are several ways to achieve this, but the easiest is to create a Federated Service and Federated ReplicaSet. To create appropriately labelled pods and services in the 13 underlying clusters of your federation:
kubectl --context=federation-cluster create -f services/nginx.yaml
kubectl --context=federation-cluster create -f myreplicaset.yaml
Note that in order for your federated ingress to work correctly on Google Cloud, the node ports of all of the underlying cluster-local services need to be identical. If you’re using a federated service this is easy to do. Simply pick a node port that is not already being used in any of your clusters, and add that to the spec of your federated service. If you do not specify a node port for your federated service, each cluster will choose its own node port for its cluster-local shard of the service, and these will probably end up being different, which is not what you want.
You can verify this by checking in each of the underlying clusters. For example:
kubectl --context=gce-asia-east1a get services nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.63.250.98 104.199.136.89 80/TCP 9m
Federations of Kubernetes Clusters can include clusters running in different cloud providers (for example, Google Cloud, AWS), and on-premises (for example, on OpenStack). However, in Kubernetes v1.4, Federated Ingress is only supported across Google Cloud clusters.
Ingress objects (in both plain Kubernetes clusters, and in federations of clusters) expose one or more IP addresses (via the Status.Loadbalancer.Ingress field) that remains static for the lifetime of the Ingress object (in future, automatically managed DNS names might also be added). All clients (whether internal to your cluster, or on the external network or internet) should connect to one of these IP or DNS addresses. All client requests are automatically routed, via the shortest network path, to a healthy pod in the closest cluster to the origin of the request. So for example, HTTP(S) requests from internet users in Europe will be routed directly to the closest cluster in Europe that has available capacity. If there are no such clusters in Europe, the request will be routed to the next closest cluster (typically in the U.S.).
Ingresses are backed by Services, which are typically (but not always) backed by one or more ReplicaSets. For Federated Ingresses, it is common practise to use the federated variants of Services and ReplicaSets for this purpose.
In particular, Federated ReplicaSets ensure that the desired number of pods are kept running in each cluster, even in the event of node failures. In the event of entire cluster or availability zone failures, Federated ReplicaSets automatically place additional replicas in the other available clusters in the federation to accommodate the traffic which was previously being served by the now unavailable cluster. While the Federated ReplicaSet ensures that sufficient replicas are kept running, the Federated Ingress ensures that user traffic is automatically redirected away from the failed cluster to other available clusters.
Check that your:
kubectl
) is correctly configured (including API endpoints and login credentials).See the federation admin guide to learn how to bring up a cluster federation correctly (or have your cluster administrator do this for you), and how to correctly configure your client.
Check that:
kubectl describe clusters
)kubectl --namespace=federation logs $(kubectl get pods --namespace=federation -l module=federation-controller-manager -o name
)ingress-controller
,
service-controller
or replicaset-controller
,
errors in the output of kubectl logs federation-controller-manager --namespace federation
).Check that:
Was this page helpful?
Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on Stack Overflow. Open an issue in the GitHub repo if you want to report a problem or suggest an improvement.