PowerScale OneFS HDFS Configuration Guide

HDFS and SmartConnect

You can configure a SmartConnect DNS zone to manage connections from Hadoop compute clients.

SmartConnect is a module that specifies how the DNS server on a PowerScale cluster handles connection requests from clients. For each IP address pool on the PowerScale cluster, you can configure a SmartConnect DNS zone which is a fully qualified domain name (FQDN).

For more information about SmartConnect, refer to the PowerSCale OneFS CLI Administration Guide or PowerScale OneFS Web Administration Guide for your version of OneFS.

Note the following:

• Hadoop compute clients can connect to the cluster through the SmartConnect DNS zone name, and SmartConnect evenly distributes NameNode requests across IP addresses and nodes in the pool.
• When a Hadoop compute client makes an initial DNS request to connect to the SmartConnect zone, the Hadoop client is routed to the IP address of an PowerScale node that serves as a NameNode. Subsequent requests from the Hadoop compute client go to the same node. When a second Hadoop client makes a DNS request for the SmartConnect zone, SmartConnect balances traffic and routes the client connection to a different node than that used by the previous Hadoop compute client.
• If you specify a SmartConnect DNS zone that you want Hadoop compute clients to connect through, you must add a Name Server (NS) record as a delegated domain to the authoritative DNS zone that contains the PowerScale cluster.
• On the Hadoop compute cluster, you must set the value of the fs.defaultFS property to the SmartConnect DNS zone name in the core-site.xml file.