access error while trying to run distcp from source cluster

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

access error while trying to run distcp from source cluster

nancy henry
Hi Team,

I am trying to copy data from A cluster to B cluster and same user for both 

I am running distcp command on source cluster A

but i am getting error

17/05/25 07:24:08 INFO mapreduce.Job: Running job: job_1492549627402_344485 17/05/25 07:24:17 INFO mapreduce.Job: Job job_1492549627402_344485 running in uber mode : false 17/05/25 07:24:17 INFO mapreduce.Job:  map 0% reduce 0% 17/05/25 07:24:26 INFO mapreduce.Job: Task Id : attempt_1492549627402_344485_m_000000_0, Status : FAILED Error: org.apache.hadoop.security.AccessControlException: User abcde (user id 50006054)  has been denied access to create distcptest2         atcom.mapr.fs.MapRFileSystem.makeDir(MapRFileSystem.java:1282)         at com.mapr.fs.MapRFileSystem.mkdirs(MapRFileSystem.java:1302)         at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1913)         at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:272)         at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:51)         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)         at java.security.AccessController.doPrivileged(Native Method)         at javax.security.auth.Subject.doAs(Subject.java:415)         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
this is the error
Reply | Threaded
Open this post in threaded view
|

Re: access error while trying to run distcp from source cluster

Sidharth Kumar
Hi ,

It may be because user don't have the write permission in destination cluster path. 

For example
$Su - abcde
$hadoop  distcp /data/sample1 hdfs://destclstnn:8020/data/

So,in the above case user abcde should have the write permission at destination path hdfs://destclstnn:8020/data/


Regards
Sidharth



On 25-May-2017 10:25 PM, "nancy henry" <[hidden email]> wrote:
Hi Team,

I am trying to copy data from A cluster to B cluster and same user for both 

I am running distcp command on source cluster A

but i am getting error

17/05/25 07:24:08 INFO mapreduce.Job: Running job: job_1492549627402_344485 17/05/25 07:24:17 INFO mapreduce.Job: Job job_1492549627402_344485 running in uber mode : false 17/05/25 07:24:17 INFO mapreduce.Job:  map 0% reduce 0% 17/05/25 07:24:26 INFO mapreduce.Job: Task Id : attempt_1492549627402_344485_m_000000_0, Status : FAILED Error: org.apache.hadoop.security.AccessControlException: User abcde (user id 50006054)  has been denied access to create distcptest2         atcom.mapr.fs.MapRFileSystem.makeDir(MapRFileSystem.java:1282)         at com.mapr.fs.MapRFileSystem.mkdirs(MapRFileSystem.java:1302)         at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1913)         at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:272)         at org.apache.hadoop.tools.mapred.CopyMapper.map(CopyMapper.java:51)         at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)         at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)         at java.security.AccessController.doPrivileged(Native Method)         at javax.security.auth.Subject.doAs(Subject.java:415)         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)         at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
this is the error