HDFS Rack Awareness

Apache Hadoop supports a feature called Rack Awareness, which allows users to define a topology for the nodes making up a cluster. Hadoop will then use that topology to spread out replicas of blocks in a fashion that maximizes fault tolerance.

The default write path, for example, is to put replicas of a newly created block first on a different node, but within the same rack, and the second copy on a node in a remote rack. In order for this to work properly, Hadoop needs to have access to the information about the underlying infrastructure it runs on. In a Kubernetes environment, this means obtaining information from the pods or nodes of the cluster.

In order to enable gathering this information the Hadoop images contain https://github.com/stackabletech/hdfs-topology-provider on the classpath, which can be configured to read labels from Kubernetes objects.

In the current version of the SDP this is now exposed as fully integrated functionality in the operator, and no longer needs to be configured via config overrides.

Prior to SDP release 24.3, it was necessary to manually deploy RBAC objects to allow the Hadoop pods access to the necessary Kubernetes objects. This ClusterRole allows the reading of pods and nodes and needs to be bound to the individual ServiceAccounts that are deployed per Hadoop cluster: this is now performed by the operator itself.

Configuration of the tool is done by using the field rackAwareness under the cluster configuration:

apiVersion: hdfs.stackable.tech/v1alpha1
kind: HdfsCluster
metadata:
  name: simple-hdfs
spec:
  clusterConfig:
    rackAwareness:
      - nodeLabel: topology.kubernetes.io/zone
      - podLabel: app.kubernetes.io/role-group
  nameNodes:
    ...

Internally this will be used to create a topology label consisting of the value of the node label topology.kubernetes.io/zone and the pod label app.kubernetes.io/role-group, e.g. /eu-central-1/rg1.