Hadoop3.0.0-alpha2-Docker run time launch errors.

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Hadoop3.0.0-alpha2-Docker run time launch errors.

Jasson Chenwei
hi, all

I have problems launching docker container in Hadoop3.0.0-alpha2/3.

I found applications failed to start during initializing the docker container. That's caused by the docker can not find launch_contaienr.sh in its workdir.

Here is my log: 
2017-05-24 11:03:09,662 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: private script path: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/nmPrivate/application_1495644587391_0001/container_1495644587391_0001_02_000001/launch_container.sh 
2017-05-24 11:03:09,695 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: private token path: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/nmPrivate/application_1495644587391_0001/container_1495644587391_0001_02_000001/container_1495644587391_0001_02_000001.tokens 
2017-05-24 11:03:09,837 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: private jar path: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/nmPrivate/application_1495644587391_0001/container_1495644587391_0001_02_000001 
2017-05-24 11:03:09,876 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1495644587391_0001_02_000001 transitioned from SCHEDULED to RUNNING 
2017-05-24 11:03:09,876 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1495644587391_0001_02_000001 
2017-05-24 11:03:09,876 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime: container working dir: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/usercache/cwei/appcache/application_1495644587391_0001/container_1495644587391_0001_02_000001 


Since ContainerLaunch will write the launch script as well as token file to 
/yarn-temp/nm-local-dir/nmPrivate/application_id/container_id. 

However, the docker working dir is 
/yarn-temp/nm-local-dir/usercache/user/appcache/application_id/container_id. It's a miss match that docker can not locate the launch script. 


I found launch script is initially written to a temporary directory, and finally should be copied to workdir. This logic is implemented in DefaultContainerExecutor, but did not find why it is not implemented in LinuxContainerExecutor.


I am not sure if this is a configuration issue. Appreciate any help who have ever tried docker container runtime on Hadoop-3.0.0-alpha2/3.



Wei 
Loading...