ECS 3.6.2 Data Access Guide

Accessing the bucket as a file system

The HDFS file system storage is provided by an ECS bucket. When you create a bucket, you must configure it in ECS so that it is available as a file system.

ECS (through the ECS Client Library) uses the permissions configured against the bucket and the settings in the Hadoop core-site.xml file to determine access to the root file system (bucket). You must ensure that you have configured sufficient access to enable Hadoop users and services to create files and directories in the bucket.

In general, all file and directory operations must be permitted by the bucket ACLs. Additionally, each individual file and directory object within the bucket has its own object ACL and all object operations must also be permitted by the object ACL. If the object operation does not satisfy the bucket ACL, the operation is denied. If the object operation does not satisfy the object ACL, the operation is denied.

An exception to this is that the bucket owner and the Hadoop superuser and members of the Hadoop supergroup, defined in hdfs-site.xml, are always permitted to perform any file system operation regardless of bucket and object ACLs.

You can set bucket ACLs by explicitly adding user ACLs on the bucket for every user, or by specifying custom group ACLs. For more information, see Bucket Custom Group ACLs and Default Group. The bucket owner must be an ECS object user. Other users do not need to be ECS object users and can be UNIX usernames from the Hadoop cluster.

A further exception is that, unlike normal ECS buckets, a file system-enabled ECS bucket has a special object that represents the root directory and a special object for each directory. The root directory object does not exist in a new file system-enabled bucket does not have a root directory object, but is created when the first file system operation is performed on the bucket. When such a root directory object exists, some ECS HDFS API calls do not perform bucket ACL checks.

To ensure consistent permissions regardless of the API call, you should ensure that the root directory object ACL duplicates the bucket ACL.

Once users have access to the file system, the files and directories that they create have permissions determined by the umask property in the core-site.xml file.