ECS 3.6.2 Data Access Guide

Migration from a simple to a Kerberos Hadoop cluster

ECS provides support for migrating from a simple Hadoop environment to a Hadoop environment secured by Kerberos.

When ECS HDFS is integrated with a Hadoop environment that uses simple security, files and directories created by Hadoop users, and processes, will be owned by non-secure users. If you subsequently migrate the Hadoop cluster to use Kerberos security, the files and directories written to ECS HDFS will no longer be accessible to those users.

ECS provides a built-in migration feature that enables you to provide ECS with a mapping between shortnames and Kerberos principals, so that files owned by non-secure shortnames will be accessible as the mapped Kerberos principal.

Where you only have a small number of files that have been written by shortname users, you might want to change them (using chown) to be owned by the Kerberos principal. However, where you have a large number of files, the migration feature means you do not have to change their ownership.

This feature is not implemented for buckets and you must change the bucket ACLs to allow access by the Kerberos principals if you are relying on access by users. However, if you use group membership as the primary means for enabling access, you do not have to change the bucket ACLs.

ECS allows the use of groups to simplify access to buckets, files, and directories. Groups always use UNIX simple names, so the group name associated with a bucket, file or directory is the same when accessing them from a simple or Kerberized cluster. When accessing from a simple environment, group membership is determined from the UNIX machine. When accessing from a Kerberized cluster you can configure group membership by assigning the mapping. Refer to Map group names for information on mapping group names.

When using AD credentials, the mapping between AD principals and UNIX principals is achieved by removing the domain suffix, so user hdfs@domain.com becomes hdfs. This is not quite as flexible as when using Kerberos principal mapping which allow mappings such as hdfs-xx@realm.com to hdfs.

When using groups with AD, an authentication provider must have been configured in ECS so that membership of the group can be checked.