- Notes, cautions, and warnings
- S3
- Amazon S3 API support in ECS
- S3 API supported and unsupported features
- Bucket policy support
- Object Tagging
- S3 Object Lock
- Object lifecycle management
- S3 Extensions
- Metadata Search
- S3 and Swift Interoperability
- Create and manage secret keys
- Authenticating with the S3 service
- Using s3curl with ECS
- Use SDKs to access the S3 service
- ECS S3 error codes
- Hadoop S3A for ECS
- Enabling data2 IP in ECS S3
- Cloud DVR
- ECS IAM for S3
- OpenStack Swift
- EMC Atmos
- CAS
- Setting up CAS support in ECS
- Cold Storage
- Compliance
- CAS retention in ECS
- Advanced retention for CAS applications: event-based retention, litigation hold, and the min/max governor
- Set up namespace retention policies
- Create and set up a bucket for a CAS user
- Set up a CAS object user
- Set up bucket ACLs for CAS
- ECS Management APIs that support CAS users
- Content Addressable Storage (CAS) SDK API support
- ECS CAS error codes
- Enabling data2 IP in CAS
- ECS Management REST API
- ECS HDFS
- ECS HDFS Introduction
- Configuring Hadoop to use ECS HDFS
- Hadoop authentication modes
- Migration from a simple to a Kerberos Hadoop cluster
- File system interaction
- Supported Hadoop applications
- Integrate a simple Hadoop cluster with ECS HDFS
- Install Hortonworks HDP using Ambari
- Create a bucket for HDFS using the ECS Portal
- Plan the ECS HDFS and Hadoop integration
- Obtain the ECS HDFS installation and support package
- Deploy the ECS HDFS Client Library
- Configure ECS client properties
- Set up Hive
- Verify Hadoop access to ECS
- Secure the bucket
- Relocate the default file system from HDFS to an ECS bucket
- Integrate a secure Hadoop cluster with ECS HDFS
- Verify that AD/LDAP is correctly configured with a secure Hadoop cluster
- Pig test fails: unable to obtain Kerberos principal
- Permission denied for AD user
- Permissions errors
- Failed to process request
- Enable Kerberos client-side logging and debugging
- Debug Kerberos on the KDC
- Eliminate clock skew
- Configure one or more new ECS nodes with the ECS service principal
- Workaround for Yarn directory does not exist error
- Set up the Kerberos KDC
- Configure AD user authentication for Kerberos
- Secure bucket metadata
- Hadoop core-site.xml properties for ECS HDFS
- Hadoop core-site.xml properties for ECS S3
- External key management
- Document feedback
ECS 3.6.2 Data Access Guide
S3 API supported and unsupported features
ECS supports a subset of the Amazon S3 REST API.
The following sections detail the supported and unsupported APIs:
Supported S3 APIs
| Feature | Notes |
|---|---|
| GET Service | ECS supports marker and max-keys parameters to enable paging of bucket list.
GET /?marker=<bucket>&limit=<num>For example: GET /?marker=mybucket&limit=40 |
| DELETE Bucket | - |
| DELETE Bucket cors | - |
| DELETE Bucket life cycle | Only the expiration part is supported in life cycle. Policies that are related to archiving (AWS Glacier) are not supported. Lifecycle is not supported on file system-enabled buckets. |
| DELETE Bucket policy | - |
| GET Bucket (List Objects) |
|
| GET Bucket (List Objects) Version 2 |
|
| GET Bucket cors | - |
| GET Bucket acl | - |
| GET Bucket life cycle | Only the expiration part is supported in life cycle. Policies that are related to archiving (AWS Glacier) are not supported. Lifecycle is not supported on file system-enabled buckets. |
| GET Bucket policy | - |
| GET Bucket Object versions | - |
| GET Bucket versioning | - |
| HEAD Bucket | - |
| List Multipart Uploads | - |
| PUT Bucket | Where PUT is performed on an existing bucket, refer to Behavior where bucket already exists. |
| PUT Bucket cors | - |
| PUT Bucket acl | - |
| PUT Bucket life cycle | Only the expiration part is supported in life cycle. Policies that are related to archiving (AWS Glacier) are not supported. Lifecycle is not supported on file system-enabled buckets. |
| PUT Bucket policy | Cannot configure the bucket policies for file system-enabled or CAS-enabled buckets. Cannot configure the bucket policies for operations that ECS does not support. More information about bucket policy support is provided in Bucket policy support. |
| PUT Bucket versioning | - |
| DELETE Object | - |
| Delete Multiple Objects | - |
| GET Object | - |
| GET Object ACL | - |
| HEAD Object | - |
| PUT Object | Supports chunked PUT |
| PUT Object acl | - |
| PUT Object - Copy | If the object size is greater than 5 GB, use Upload Part - Copy instead of PUT Object - Copy. |
| OPTIONS object | - |
| GET Object tagging | - |
| PUT Object tagging | - |
| DELETE Object tagging | - |
| Initiate Multipart Upload | - |
| Upload Part | - |
| Upload Part - Copy | - |
| Complete Multipart Upload | ECS returns an ETag of 00 for this request, which differs from the Amazon S3 response. |
| Abort Multipart Upload | - |
| List Parts | - |
| PUT Bucket Object Lock | - |
| GET Bucket Object Lock | - |
|
PUT Bucket enable-object-lock |
This is a custom API (not defined by S3) |
| PUT Object legal hold | - |
| GET Object legal hold | - |
| PUT Object retention | - |
| GET Object retention | - |
NOTE:
- Creation of buckets using names with fewer than three characters fails with 400 Bad Request, InvalidBucketName.
- When creating a bucket or object with empty content, ECS returns 400 invalid content-length value, which differs from AWS which returns 400 Bad Request.
- Copying an object to another bucket that indexes the same user metadata index key but with a different datatype is not supported and fails with 500 Server Error.
- When listing the objects in a bucket, if you use a prefix and delimiter but supply an invalid marker, ECS throws 500 Server Error, or 400 Bad Request for a file system-enabled bucket. However, AWS returns 200 OK and the objects are not listed.
-
For versioning enabled buckets, ECS does not create a delete marker when a deleted object is deleted again. This is different from AWS, which always inserts delete marker for deleting deleted objects in versioning enabled buckets. This change in behavior is only applicable when the deleted object is deleted again from owner zone.
| Feature | Notes |
|---|---|
| Presigned URLs | ECS supports use of presigned URLs to grant access to objects without needing credentials. More information can be found at: https://docs.aws.amazon.com/AmazonS3/latest/dev/PresignedUrlUploadObject.html. |
| Chunked PUT | PUT operation can be used to upload objects in chunks, which enable content to be sent before the total size of the payload is known. Chunked transfer uses the Transfer-Encoding header (Transfer-Encoding: chunked) to specify that content is transmitted in chunks. |
Unsupported S3 APIs
| Feature | Notes |
|---|---|
| DELETE Bucket tagging | - |
| DELETE Bucket website | - |
| GET Bucket location | ECS is only aware of a single Virtual Data Center (VDC). |
| GET Bucket logging | - |
| GET Bucket notification | Notification is only defined for reduced redundancy feature in S3. ECS does not support notifications. |
| GET Bucket tagging | - |
| GET Bucket requestPayment | ECS uses its own model for payments. |
| GET Bucket website | - |
| PUT Bucket logging | - |
| PUT Bucket notification | Notification is only defined for the reduced redundancy feature in S3. ECS does not support notifications. |
| PUT Bucket tagging | - |
| PUT Bucket requestPayment | ECS uses its own model for payments. |
| PUT Bucket website | - |
| Object APIs | |
| GET Object torrent | - |
| POST Object | - |
| POST Object restore | The POST Object restore operation is related to AWS Glacier, which is not supported in ECS. |
| SELECT Object Content | - |
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please select whether the article was helpful or not.
Comments cannot contain these special characters: <>()\