Skip to main content
  • Place orders quickly and easily
  • View orders and track your shipping status
  • Enjoy members-only rewards and discounts
  • Create and access a list of your products

ECS:NFS 遠端 I/O 錯誤;變更啟用 FS 容器的容器擁有者可能會導致應用程式及/或使用者無法存取 NFS 檔案

Summary: 前一代儲存器擁有者不允許或限制 ObjectControllerException:前一個儲存器擁有者不允許的 UpdateObjectInternal 方法

This article applies to   This article does not apply to 

Symptoms

由使用者變更為使用者介面上的儲存器擁有者頁面:
kA23a0000000BOMCA2_3_0

此問題適用于啟用 NFS 的儲存器,以及使用者介面變更儲存器擁有者。這可能會導致已連線的應用程式或使用者無法存取 Linux 檔案系統上的儲存器。即使我們再次將變更還原至原始擁有者,也無法再次存取,導致 DU。

在此範例中:
儲存器擁有者已使用使用者介面變更為「sham2」。由於 ECS 的限制,甚至在將儲存器擁有者名稱變更為「sham1」之後仍如此。ECS 不會使用使用者介面將儲存器擁有者變更為「sham1」。現在只能在 CLI 使用具有負載的 API,將系統旗標重新設定為 true 時,才能完成此作業。

在 Linux 機器上找出問題的方法,請使用者觸碰檔案,例如:
admin@node1~>touch file
touch: setting times of `file': Remote I/O error

se svc_log with the string  "method updateObjectInternal " 

Command:
# svc_log -a -sr dataheadsvc | grep "method updateObjectInternal"
Example:

admin@node1~>svc_log -a -sr dataheadsvc | grep "method updateObjectInternal"
svc_log v1.0.22 (svc_tools v1.5.3)                 Started 2019-06-06 10:45:04

Running on nodes:              <All nodes>
Time range:                    2019-06-05 10:45:04 - 2019-06-06 10:45:04
Filter string(s):              <All messages>
Show nodename(s):              True
Search reclaim logs (if any):  False

com.emc.storageos.data.object.exception.ObjectControllerException: method updateObjectInternal not allowed for previous bucket owner sham1
Caused by: com.emc.storageos.data.object.exception.ObjectControllerException: method updateObjectInternal not allowed for previous bucket owner sham1

Cause

以特定使用者作為擁有者建立儲存器,然後變更儲存器的擁有權。最後,讓原始擁有者使用 ACL 頁面完全掌控失敗,並顯示 ECS 記錄例外狀況:
ObjectControllerException: method updateObjectInternal not allowed for previous bucket owner <ownerid>
This is a known issue currently being evaluated by Dell EMC at this time.

Resolution

因應措施是透過含負載的 CLI 使用 API 將儲存器擁有者變更為 true。

1.決定目前的儲存器擁有者。
要求使用者介面根密碼產生 TOKEN。例如:
admin@ecsnode1:~> tok=$(curl -iks https://XX.XX.XX.XX:4443/login -u 'root:ChangeMe' | grep X-SDS-AUTH-TOKEN)

確認目前的儲存器擁有者 (在您的情況下,替換儲存器和命名空間):
admin@node1:~> curl -s -k -X GET -H "$tok" https://XX.XX.XX.XX:4443/object/bucket/sham_bk_nfs/info?namespace=degreat_nfs | xmllint --format - | grep '<owner>'
  <owner>sham2</owner>

這會確認必須將reset_previous_owners參數設為 true。還原的儲存器擁有者位於使用者介面上,但透過命令列介面 (CLI) 的 API 可確認 ECS 仍將儲存器擁有者視為「sham2」。

2.使用 vi 編輯器建立簡單的 xml 檔案。在下方的範例中,稱為/tmp/bucket-owner.xml。這是兩個步驟的程式。我們必須暫時將它設為新的擁有者。第二個步驟。如以下範例所示,請先確認輸出,再重新設定為原始擁有者,並確認輸出:
admin@node1:~ # vi /tmp/bucket-owner.xml

admin@ecsnode1:~ # cat /tmp/bucket-owner.xml
<object_bucket_update_owner>
   <namespace>degreat_nfs</namespace>
   <new_owner>sham2</new_owner>
   <reset_previous_owners>true</reset_previous_owners>
</object_bucket_update_owner>

3. Change the bucket owner to the temporary owner. 

透過 xml 檔案將儲存器擁有者變更為「sham2」所需的 API 語法如下:
admin@ecsnode1:~> curl -v -k -X "POST" "https://xx.xx.xx.xx:4443/object/bucket/sham_bk_nfs/owner" -H "$tok" -H "Content-Type: application/xml" -H "ACCEPT:application/xml" -d @/tmp/bucket-owner.xml -v
* Hostname was NOT found in DNS cache
*   Trying xx.xx.xx.xx...
* Connected to xx.xx.xx.xx (xx.xx.xx.xx) port 4443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs/
* SSLv3, TLS unknown, Certificate Status (22):
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, Certificate (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* Server certificate:
*        subject: CN=localhost
*        start date: 2019-03-25 09:53:41 GMT
*        expire date: 2029-03-22 09:53:41 GMT
*        issuer: CN=localhost
*        SSL certificate verify result: self signed certificate (18), continuing anyway.
> POST /object/bucket/sham_bk_nfs/owner HTTP/1.1
> User-Agent: curl/7.37.0
> Host: xx.xx.xx.xx:4443
> X-SDS-AUTH-TOKEN: BAAcUy9KYlhxTlVYb2M0bnF3bTNscEsvSEdDeWhJPQMAjAQASHVybjpzdG9yYWdlb3M6VmlydHVhbERhdGFDZW50ZXJEYXRhOmJhOGQ3ZTkzLTMyMGYtNDNmNy05Y2FkLWM4YWQzMWFiMzY1MAIADTE1NTk3Mzk3OTA2MDgDAC51cm46VG9rZW46YjQ4NGNiZjEtNTkwNy00YWI3LTgzYTctM2Y3OGRhM2RiY2NiAgAC0A8=
> Content-Type: application/xml
> ACCEPT:application/xml
> Content-Length: 179
>
* upload completely sent off: 179 out of 179 bytes
< HTTP/1.1 200 OK
< Date: Thu, 06 Jun 2019 10:56:08 GMT
< Content-Length: 0
< Connection: keep-alive
<
* Connection #0 to host xx.xx.xx.xx left intact
4. Edit the simple.xml file previously created in step 2 and this time insert original owner of sham1

admin@node1:~ # vi /tmp/bucket-owner.xml

admin@ecsnode1:~ # cat /tmp/bucket-owner.xml
<object_bucket_update_owner>
   <namespace>degreat_nfs</namespace>
   <new_owner>sham1</new_owner>
   <reset_previous_owners>true</reset_previous_owners>
</object_bucket_update_owner>

5. Change the bucket owner back to the original owner
The API syntax required to change the bucket owner back to "sham1" through the xml file is as follows:
admin@ecsnode1:~> curl -v -k -X "POST" "https://xx.xx.xx.xx:4443/object/bucket/sham_bk_nfs/owner" -H "$tok" -H "Content-Type: application/xml" -H "ACCEPT:application/xml" -d @/tmp/bucket-owner.xml -v
* Hostname was NOT found in DNS cache
*   Trying xx.xx.xx.xx...
* Connected to xx.xx.xx.xx (xx.xx.xx.xx) port 4443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs/
* SSLv3, TLS unknown, Certificate Status (22):
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, Certificate (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* Server certificate:
*        subject: CN=localhost
*        start date: 2019-03-25 09:53:41 GMT
*        expire date: 2029-03-22 09:53:41 GMT
*        issuer: CN=localhost
*        SSL certificate verify result: self signed certificate (18), continuing anyway.
> POST /object/bucket/sham_bk_nfs/owner HTTP/1.1
> User-Agent: curl/7.37.0
> Host: xx.xx.xx.xx:4443
> X-SDS-AUTH-TOKEN: BAAcUy9KYlhxTlVYb2M0bnF3bTNscEsvSEdDeWhJPQMAjAQASHVybjpzdG9yYWdlb3M6VmlydHVhbERhdGFDZW50ZXJEYXRhOmJhOGQ3ZTkzLTMyMGYtNDNmNy05Y2FkLWM4YWQzMWFiMzY1MAIADTE1NTk3Mzk3OTA2MDgDAC51cm46VG9rZW46YjQ4NGNiZjEtNTkwNy00YWI3LTgzYTctM2Y3OGRhM2RiY2NiAgAC0A8=
> Content-Type: application/xml
> ACCEPT:application/xml
> Content-Length: 179
>
* upload completely sent off: 179 out of 179 bytes
< HTTP/1.1 200 OK
< Date: Thu, 06 Jun 2019 10:56:08 GMT
< Content-Length: 0
< Connection: keep-alive
<
* Connection #0 to host xx.xx.xx.xx left intact


6.  Confirm the bucket owner change is reflected. 

確認儲存器擁有者變更現在是「sham1」。
admin@ecsnode1:~> curl -s -k -X GET -H "$tok" https://XX.XX.XX.XX:4443/object/bucket/sham_bk_nfs/info?namespace=degreat_nfs | xmllint --format - | grep '<owner>'
<owner>sham1</owner>

在 API 上還原儲存器擁有者後,請確認主機現在可以存取 Linux 檔案系統上的儲存器。

.7. 組態變更完成後,我們不應再看到相同的錯誤
svc_log -f "method updateObjectInternal not allowed" -start "20 hour ago" -sr all -sh -st hour
svc_log v1.0.22 (svc_tools v1.6.8)                 Started 2020-01-23 09:28:17

Running on nodes:              <All nodes>
Time range:                    2020-01-22 13:28:17 - 2020-01-23 09:28:17
Filter string(s):              'method updateObjectInternal not allowed'
Show nodename(s):              True
Search reclaim logs (if any):  False

Count of message occurrences per hour:

2020-01-22 13:xx - 5066
2020-01-22 14:xx - 9580
2020-01-22 15:xx - 9574
2020-01-22 16:xx - 9580
2020-01-22 17:xx - 9570
2020-01-22 18:xx - 9576
2020-01-22 19:xx - 9564
2020-01-22 20:xx - 9576
2020-01-22 21:xx - 9576
2020-01-22 22:xx - 9572
2020-01-22 23:xx - 9564
2020-01-23 00:xx - 9586
2020-01-23 01:xx - 9574
2020-01-23 02:xx - 9572
2020-01-23 03:xx - 4564
2020-01-23 04:xx - 0
2020-01-23 05:xx - 0
2020-01-23 06:xx - 0
2020-01-23 07:xx - 0
2020-01-23 08:xx - 0
2020-01-23 09:xx - 0


Dell EMC is aware of this issue and are working on a fix in a future release.

Additional Information

Affected Products

Elastic Cloud Storage

Products

ECS Appliance, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, Elastic Cloud Storage