Troubleshoot upload phase when running scripts on hosts
Overcome the
PowerFlex Gateway not being able to upload files using the "Run script on host" feature.
About this task
If the "Upload" phase fails with a timeout error, check if the following appears in the scaleio.log file on the
PowerFlex Gateway, and the LIA log file of the relevant node. If the error appears, follow the remediation steps listed below. This issue is caused by a lack of resources in the
PowerFlex Gateway's operating system, and in the JAVA process sustaining the upload.
Typical scaleio.log file:
2020-07-30 11:35:28,333 [executor-23] INFO c.e.s.s.d.i.c.UploadFileCommand - activating serial upload force algorithm due to upload of package UploadFileCommand[9.20.25.23, 9.20.125.23] that has failed due to memory problem java.lang.OutOfMemoryError: Java heap space
2020-07-30 11:35:28,333 [executor-23] ERROR c.e.s.s.d.i.c.UploadFileCommand - UNEXPECTED Error java.lang.OutOfMemoryError: Java heap space executing command .UploadFileCommand (abort) :
Typical LIA log file:
30/07 11:51:19.035533 7f9f98115db0:liaCmd_CallCmdWithTimer:00253: Executing...
Command: chmod 744 /opt/emc/scaleio/lia/bin/patch_script
Timeout: [0]Sec (if zero - timeout is default)
30/07 11:51:19.040428 7f9f98115db0:liaNet_RecvRequest:00427: UPLOAD PACKAGE END
30/07 12:19:55.027051 7f9f98198db0:netSec_Recv_CK:00847: ERROR: Failed to read data from handle 0x7f9f981a3ce0 (ERROR)
30/07 12:19:55.027066 7f9f98198db0:netPath_RcvLoop_CK:01611: WARNING: :: Connected Live SERVER path 0x7f9f981dd950 of portal 0x7f9f981ff170 net 0xa3a3c0 socket 11 inflights 0 HS:0 socket receive failed, Status ERROR
30/07 12:19:55.027076 7f9f98198db0:netCon_AbortInflightRpcsUptoDummyNode:05869: Aborted 0 inflight RPC messages due to disconnect.
Steps
Check if the
PowerFlex Gateway's operating system has sufficient memory: minimum 8 GiB, recommended 12 GiB.
In case of SVM this requires shutdown of the SVM to modify the memory
This can be done at any time as long as no gateway orchestration is running
If "Run Script on Host" is running abort the operation prior to shutdown of the OS
After adding memory we need to increase the Java settings:
Using file editor (such as vi) open the file: /opt/emc/scaleio/gateway/bin/setenv.sh and modify the following line:
From:
CATALINA_OPTS="-Xms128m -Xmx2048m"
To:
CATALINA_OPTS="-Xms128m -Xmx4096m"
Restart the gateway service:
Linux:
systemctl scaleio-gateway restart
Windows: Restart the service from the
services.msc window.
Log in to the
PowerFlex Gateway, and run the "Run Script on Host" feature with upload capability again.
Data is not available for the Topic
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please provide ratings (1-5 stars).
Please select whether the article was helpful or not.
Comments cannot contain these special characters: <>()\