Skip to main content
Sign Out

Welcome to Dell

My Account
  • Place orders quickly and easily
  • View orders and track your shipping status
  • Create and access a list of your products
Sign In Create an Account Premier Sign In Partner Program Sign In
Contact Us

Your Dell.com Carts

Some article numbers may have changed. If this isn't what you're looking for, try searching all articles. Search articles

ECS: Bucket Management - Overview and Troubleshooting

Summary: This article describes how to manage buckets, considerations and troubleshooting.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

This article describes buckets, bucket management and basic troubleshooting tips.

Cause

N/A

Resolution

Bucket Management

What is a bucket?

Buckets are containers for objects created in a namespace and sometimes considered a logical container for sub-tenants. In S3, containers are called buckets and this term has been adopted by ECS. In Atmos, the equivalent of a bucket is a subtenant; in Swift, the equivalent of a bucket is a container, and for CAS, a bucket is a CAS pool. Buckets are global resources in ECS. Each bucket is created in a namespace and each namespace is created in a Replication Group (RG).

Buckets are global resources that can span multiple sites. Bucket creation involves assigning it to a namespace and a RG. The bucket level is where ownership and file or CAS access is enabled.

Buckets can be accessed via different tools at the same time, i.e. access the same bucket with GeoDrive and S3 Browser.


How to create a bucket?

ECS offers several ways to create a bucket. To create a bucket the following must be done:
  1. Upload an ECS license.
  2. Select a set of nodes to create at least one storage pool.
  3. Create a VDC.
  4. Create at least one Replication Group.
  5. Create at least one namespace.
Bucket creation can be done via the ECS GUI or Rest API:
  • ECS GUI
Log in to the ECS system using your browser and select the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide - 'Getting started with ECS - 'Create a bucket'
  • Rest API
Refer to the ECS REST API guide for management and object operations.


What protocols are supported?

When a object user is created - 'Manage' - 'Users' - 'New Object User', the following protocol options are available:
  • S3 
  • Atmos
  • Swift
  • CAS
  • File access via NFSv3 (S3)
  • File access to CIFS via GeoDrive (S3)


What bucket options are supported? - Bucket creation

By creating a new bucket - 'Manage' - 'Buckets' - 'New Bucket', the following options are available:
 
File system
Required for NFSv3. Required on bucket creation. Note this cannot be altered once the bucket is created.

Refer to article ECS: Best practices for mounting NFS exports for further information.
CAS
For CAS operations, this must be set when creating the bucket. CAS buckets cannot be accessed with other protocols and the CAS option must be set when creating a bucket.

Refer to article ECS: CAS User & Bucket Administration Best Practices for further information.
Metadata Search
Enable indexing of Objects based on user and system metadata, and cannot be enabled after save.

Refer to the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide  - 'Buckets' - 'Working with buckets in the ECS Portal' - 'Bucket settings' - 'Metadata search fields'
Access during outage (ADO)
ECS allows you to configure buckets to be accessible if one of your VDC fails.

Refer to article ECS: The importance of the ADO (Access During Outage) parameter for further information.
Server-side Encryption
Enable to allow data-at-rest encryption, which encrypts data on the storage disks to reduce the risk of data loss by physical theft of disks.

Refer to the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide  - 'Buckets' - 'Working with buckets in the ECS Portal' - 'Bucket settings' 
Quota
Soft and Hard Quota can be assigned on either, bucket or namespace level.

Refer to article ECS: How to increase the quota set on an ECS Bucket for further information.
Bucket Tagging
Bucket tags are key-value pairs that you can associate with a bucket. Max 10 tags per bucket.

Refer to the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide  - 'Buckets' - 'Working with buckets in the ECS Portal' - 'Bucket settings' - 'Bucket tagging'
Retention
Retention can be set on bucket or namespace level. Retention applies to CAS and S3 buckets.

Refer to the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide - 'Namespaces' - 'Working with namespaces in the ECS Portal' - 'Namespace settings' - 'Retention periods and policies'


What bucket options are supported? - Bucket edit

Once a bucket is created, there is an option to modify bucket settings via the 'Edit Bucket' drop down with 'Edit ACL' and 'Edit Policy'. Refer to the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide - 'Manage' - 'Buckets' - 'Edit Bucket'.

The privileges a user has when accessing a bucket are set using an Access Control List (ACL). You can assign ACLs for a user, for a set of pre-defined groups, such as all users, and for a custom group.When you create a bucket and assign an owner to it, an ACL is created that assigns a default set of permissions to the bucket owner - the owner is, by default, assigned full control. You can modify the permissions assigned to the owner or you can add new permissions for a user by selecting the 'Edit ACL' operation for the bucket:
 
Edit ACLs
S3 allows to set the following permissions:
 
Read Allows user to list the objects in the bucket.
Read ACL Allows user to read the bucket ACL.
Write Allows user to create or update any object in the bucket.
Write ACL Allows user to write the ACL for the bucket.
Execute Sets the execute permission when accessed as a file system. This permission has no effect when the object is accessed using the ECS object protocols.
Full Control Allows user to Read, Write, Read ACL, and Write ACL.
Note: Non object owners can Read, Write, Read ACL, and Write ACL if the permission has been granted or can only list the objects.
Privileged Write Allows user to perform writes to a bucket or object when the user does not have normal write permission. Required for CAS buckets.
Delete Allows user to delete buckets and objects. Required for CAS buckets.
None User has no privileges on the bucket.

Multiple users can be created with different permissions. Refer to the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide - 'Buckets' - 'Working with buckets in the ECS Portal' - 'Set ACLs'

For CAS refer to article ECS: Set up bucket ACLs for CAS for further information.
Bucket policies provide greater flexibility than ACLs and allow fine grained control over permissions for bucket operations and for operations on objects within the bucket. Policy conditions are used to assign permissions for a range of objects that match the condition and are used to automatically assign permissions to newly uploaded objects. You can modify the permissions assigned to objects or you can add new permissions by selecting the 'Edit Policy' operation for the bucket:
 
Edit Policy

The ECS GUI provides a Bucket Policy Editor to create a bucket policy for an existing bucket. Policies are defined in JSON format and the syntax used for policies is the same as that used for Amazon AWS:

kA23a000000GG6FCAW_1_0
You can use bucket policies in the following typical scenarios:

  • Grant bucket permissions to a user
  • Grant bucket permissions to all users
  • Automatically assign permissions to created objects

For examples on the above operations, refer to the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide  - 'Buckets' - 'Working with buckets in the ECS Portal' - 'Set bucket policies' - 'Bucket policy scenarios'.


How to test buckets?

There are various ways to test S3 connectivity, these are a few examples for accessing them. Simplest method to access a S3 bucket via S3 Browser.
 
WinSCP:   ECS: Connecting ECS buckets with WINSCP
s3test.sh script:  ECS: How to test S3 create bucket, upload object, read object, delete object, delete bucket with the s3test.sh script
S3curl:     ECS: How to perform basic s3 operations on ECS 3.x using the s3curl.pl script.
S3Browser:   ECS : How to connect ECS with S3Browser?


What are the bucket limitations?

Buckets are limited by the maximum capacity configured to the system the bucket is configured on.

Buckets are multi-protocol enabled, except CAS buckets. CAS buckets cannot be shared.

Client access and API operation availability at the namespace, bucket and object levels during a TSO is governed the following ADO options set at the namespace and bucket level:
  • Off (default) - Strong consistency is maintained during a temporary outage.
  • On - Eventually consistent access is allowed during a temporary site outage.
Data consistency during a TSO is implemented at the bucket level. Configuration is set at the namespace level, which sets the default ADO setting in place for ADO during new bucket creation and can be overridden at new bucket creation; meaning TSO can be configured for some buckets and not for others.


How to delete a bucket?

To delete a bucket, make sure the bucket is entirely empty. This not only includes current objects, also, if configured, versions and delete markers need to be deleted before you can remove that bucket from.

If it is not possible to delete the bucket manually, a lifecycle policy can be set on the bucket to delete it. Refer to article ECS: How to delete a S3 bucket with many objects in it using lifecycle policies for further information.

Additional Information

Considerations

Before creating a buckets remember that some options cannot be changed later:
  • Replication Group
  • Server-side Encryption
  • File System
  • CAS
  • Metadata Search
Make sure the buckets are configured properly, else a migration might be required.

The following rules apply to the naming of S3 buckets in ECS:
  • Must be between one and 255 characters in length. (S3 requires bucket names to be 1 255 characters long)
  • Can include dot (.), hyphen (-), and underscore (_) characters and alphanumeric characters ([a-zA-Z0-9])
  • Can start with a hyphen (-) or alphanumeric character.
  • Cannot start with a dot (.)
  • Cannot contain a double dot (..)
  • Cannot end with a dot (.)
  • Must not be formatted as IPv4 address.
  • Bucket names must be unique within a namespace.
  • Namespace and bucket names should be DNS compatible since they can appear in a DNS record.

Bucket best practices:
  • Use buckets for specific environment, workflow, or uses. For instance: dev, test, finance, operations, etc.
  • In multi-site deployments, create buckets at the VDC site closest to the application accessing and updating the objects. There is overhead involved with checking the latest copy if the ownership of object is at a remote site.
  • For best performance, recommended to have less than 1000 buckets in a single namespace

For additional information, select the ECS Help ('?' icon) in the ECS GUI which links the ECS Administration Guide.

ECS Code bucket limitations

Refer to the latest ECS Release Notes for up to date fixes implemented on bucket limitations.

Troubleshooting articles

Protocol error codes

ECS: S3 error codes with possible causes
ECS: Troubleshooting S3 Read/Write/Delete issues
ECS: CAS (Centera) SDK error codes
ECS: swift authentication to keystone doesn't work

Quota

ECS: How to update Bucket Quota by management API.

Retention

ECS: CAS retention in ECS

Delete issues

ECS: Unable to delete empty CAS-enabled bucket - Error 1013 (http: 400): Bad request body. Bucket {} is not empty.
ECS: Customer is unable to delete bucket

ACLs

ECS: Blank Custom Group ACL - Bucket created using create_subtenant.sh - Atmos 
ECS: Bucket and Object ACL for a non-owner object user in S3 Browser
ECS: Read only Bucket ACL for bucket non owner object user unable to download uploaded objects in S3 Browser

Lifecycle Policy

ECS: How to enable a bucket lifecycle policy to handle delete of incomplete MPU's
ECS: PUT/GET/DELETE requests on bucket with lifecycle policy (that has no prefix) returns 500 error
ECS : Ineffective life-cycle policy when more than one policy have overlapping prefixes for different actions

MPU

ECS: How to use s3curl perform MPU(Multi-part upload)

TSO

ECS: xDoctor: RAP051: TSO (temporary site outage) detected.

Base URL

ECS: Can buckets be accessed using the baseURL via a web browser?

Affected Products

ECS Appliance
Article Properties
Article Number: 000055896
Article Type: Solution
Last Modified: 19 Jul 2023
Version:  4
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.