408 Request Timeout when sending data to Isilon WebHDFS

Question

Greetings, I'm currently deploying an hortonworks lab with OneFS for HDFS stack. I have the OneFS 8.2.1 simulator release. I installed with success HDP 3.0.1, but I have un problem for initiate module like Spark2 which send at the first startup a 100MB tar file to Isilon.  The command is : curl -sS -L -w '%{http_code}' -X PUT --data-binary @/tmp/spark2/spark2-hdp-yarn-archive.tar.gz -H 'Contentpe: application/octet-str' 'http://isilonsczone-hdp.positiveyes.com:8082/webhdfs/v1/apps/spark2/spark2-hdp-yarn-archive.tar.gz?op=CREATE&user.name=hdfs&overwrite=True&permission=444 I tried to play with curl option as max-time but it's not working yet. I have the following answer : 408 Request Timeout Request Timeout Server timeout waiting for the HTTP request from the client. So I transfer the file with hadoop fs -copyFromLocal : hadoop fs -copyFromLocal /tmp/spark2/spark2-hdp-yarn-archive.tar.gz hdfs://isilonsczone-hdp:8020/hdp/apps/3.0.1.0-187/spark2/ The operation is successfull but it took at least 2 minutes (non production environment...) To be sure it was not another problem than transfer time I tried curl transfer with a simple text file and it worked  like a charm : curl -sS -L -w '%{http_code}' -X PUT --data-binary @/tmp/spark2/test/txt -H 'Content-Type: application/octet-str' 'http://isilonsczone-hdp.positiveyes.com:8082/webhdfs/v1/apps/spark2/test.txt?op=CREATE&user.name=hdfs&overwrite=True&permission=444' I tried to change apache conf on my Isilon Cluster (/etc/mcp/template/apache24.conf + webui_httpd.conf) with : TimeOut 600 +           # RequestReadTimeout header=10-60,MinRate=1024 body=30-120,MinRate=1024              RequestReadTimeout phase=0 and than restart WebUI through isi services -a isi_webui disable and enable But nothing change, I have 408 timeout error once again. I analyzed /var/log/apache log but nothing interresting. I'm really grateful for all kind of help you can provide Benoit

Phil.Lam · Answer

@benoitvillage  Have you checked this out? http://www.unstructureddatatips.com/hadoop-rest-api-webhdfs-on-onefs/

Isilon

Was this post helpful?