ECS 3.6.2 Data Access Guide

Create a bucket for HDFS using the ECS Portal

Use the ECS Portal to create a bucket configured for use with HDFS.

1. In the ECS Portal, select Manage > Buckets > New Bucket.
2. On the New Bucket page, at the Name field, enter a name for the bucket.
   NOTE:

   Do not use underscores in bucket names as they are not supported by the URI Java class. For example, viprfs://my_bucket.ns.site/ does not work as it is an invalid URI and is thus not understood by Hadoop.

3. In the Namespace field, select the namespace that the bucket will belong to.
4. In the Replication Group field, select a replication group or leave this field blank to use the default replication group for the namespace.
5. In the Bucket Owner field, type the name of the bucket owner.
   For a HDFS bucket, the bucket owner will usually be hdfs, or hdfs@REALM.COM for Kerberos buckets. The Hadoop hdfs user requires superuser privileges on the HDFS; this can be achieved by making hdfs the owner of the bucket. Other Hadoop users may also require superuser privileges and these privileges are granted by assigning users to a group and making that group a superuser group.
6. Do not turn on CAS.
   NOTE: A bucket that is intended for use as HDFS cannot be used for CAS. The CAS field is turned off when File System is turned on.
7. Turn on any other bucket features that you require.
   You can turn on the following features for a HDFS bucket:

   • Quota
   • Server-side Encryption
   • Metadata Search
   • Access During Outage
   • Compliance (see note)
   • Bucket Retention

   For information on each of these settings and how to configure them, refer to the ECS Administration Guide which is available from the .https://www.dell.com/support/.

   NOTE:

   A bucket that is compliance-enabled cannot be written to using the HDFS protocol. However, data written using object protocols can be read from HDFS.

8. In the File System field, click On.
   Once this feature is turned on, the fields for setting a default group for the file system/bucket and for assigning group permissions for files and directories created in the bucket are available.
9. In the Default Bucket Group field, type a name for the default bucket group.
   This group is the group associated with the HDFS root file system and enables Hadoop users who are members of the group to access the HDFS.
   The default group could be a group, such as hdfs or hadoop to which the services that you need to access the data on the HDFS belong, but it can be whatever group name makes sense for your Hadoop configuration. For example, the administrator might want all S3 files uploaded to the bucket to be assigned to group S3DataUsers. All S3 files will then have this group assigned to them. On the Hadoop node, the Hadoop administrator will have users that are members of the group S3DataUsers. S3DataUsers can be a Linux group, or an AD group. When the Hadoop users want to access the S3 data, they can do so because the files were uploaded and assigned to that group
   You must specify the default group at bucket creation. If you do not, the group must be assigned later from Hadoop by the file system owner.
10. In the Group File Permissions and Group Directory Permissions fields, set the default permissions for files and directories created in the bucket using the object protocols.
    You use these settings to apply UNIX group permissions to objects created using object protocols. These permissions apply to the HDFS group (the Default Bucket Group) when the object or directory is listed from Hadoop. For more information on setting the Default Group and permissions for the file system, see Multi-protocol (cross-head) access.
    1. In the Group File Permissions field, select the appropriate permission buttons. You normally set Read and Execute permissions.
    2. In the Group Directory Permissions field, select the appropriate permission buttons. You normally set Read and Execute permissions.
11. Click Save to create the bucket.