This topic explains about File System (FS) enablement feature.
S3 buckets can be File System (FS) enabled so that the files that are written using the S3 protocol can be read using the file protocols, such as Network File system (NFS) and Hadoop Distributed File System (HDFS), and the opposite way.
NOTE: Only the objects that are created in the file system enabled buckets, inherit group permissions from the bucket settings if those permissions are not specified.
Enabling FS access
You can enable FS access using the
x-emc-file-system-access-enabled header when creating a bucket using the S3 protocol. File system access can also be enabled when creating a bucket from the ECS Portal (or using the ECS Management REST API).
Limitation on FS support
When a bucket is FS enabled S3 life cycle management cannot be enabled.
Cross-head support for FS
Cross-head support is accessing objects that are written using one protocol using a different, ECS-supported protocol. Objects written using the S3 head can be read and written using NFS and HDFS file system protocols.
An important aspect of cross-head support is how object and file permissions translate between protocols and for file system access how user and group concepts translate between object and file protocols.
You can find more information about the cross-head support with file systems in the ECS Administration Guide which is available from the
https://www.dell.com/support/.
NFS WORM (Write Once, Read Many)
NFS data become Write Once Read Many (WORM) compliant when autocommit is implemented on it.
In detail, creating files through NFS is a multi step process. To write to a new file, NFS client first sends the CREATE request with no payload to NFS server. After receiving a response, the server issues a WRITE request. It is a problem for FS enabled buckets under retention as the file created with 0 bytes blocks any writes to it. Due to this reason, until ECS 3.3, retention on FS enabled bucket makes the whole mounted file-system read-only. There is no End of File (EOF) concept in NFS. Setting a retention for files, on the FS enabled buckets, after writing to them does not work as expected.
To remove the constraints that are placed on NFS files in a retention enabled bucket, the autocommit period is implemented on NFS data. For this reason, it is decided to introduce the autocommit period during which certain types of updates (for now identified as writes, Acl updates and deletes that are required for rsync, and rename that is required for Vim editor) are allowed, which removes the retention constraints for that period alone.
Autocommit period is a bucket property like retention period.
Autocommit period is:
Applicable only for the file system enabled buckets with retention period
Applicable to the buckets in noncompliant namespace
Applies to only requests from NFS and Atmos
Seal file
The seal file functionality helps to commit the file to WORM state when the file is written ignoring the remaining autocommit period. The seal function is performed through the command:
chmod ugo-w <file> on the file.
NOTE: The seal functionality does not have any effect outside the retention period.
High-level overview
Table 1. Autocommit termsThis table describes the Autocommit terms.
Term
Description
Autocommit period
Time interval relative to the object's last modified time during which certain retention constraints (example: file modifications, file deletions, and so on) are not applied. It does not have any effect outside of the retention period.
Retention Start Delay
Atmos head uses the start delay to indicate the autocommit period.
The following diagram provides an overview of the autocommit period behavior.
Autocommit configuration
The autocommit period can be set from the user interface or bucket REST API or S3 head or Atmos subtenant API.
User Interface
The user interface has the following support during bucket create and edit:
When the File System is not enabled, no autocommit option is displayed.
When the File System is enabled /no retention value that is specified, autocommit is displayed but disabled.
When the File System is enabled/retention value selected/autocommit is displayed and enabled for selection.
NOTE: Maximum autocommit period is limited to the smaller of the Bucket Retention period or the default maximum period of one day.
REST API
Create bucket REST API is modified with the new header,
x-emc-autocommit-period.
lglou063:~ # curl -i -k -T /tmp/bucket -X POST https://10.247.99.11:4443/object/bucket -H "$token" -H "Content-Type: application/xml" -v
The contents of /tmp/bucket
<object_bucket_create>
<name>bucket2</name>
<namespace>s3</namespace>
<filesystem_enabled>true</filesystem_enabled>
<autocommit_period>300</autocommit_period>
<retention>1500</retention>
</object_bucket_create>
S3 head
Bucket creation
Bucket creation flow through s3 head can make use of optional request header,
x-emc-auto-commit-period:seconds to set the autocommit period. The following checks are made in this flow: