Mismatching permissions between HDFS and HDFS NFS gateway

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Mismatching permissions between HDFS and HDFS NFS gateway

Reed Villanueva

Using HDFS NFS gateway with HDP 3.1 and seeing odd behavior where directories accessed via NFS mount have permissions different that those of the local host user that created them (eg. via "hadoop fs -mkdir ..." command). Eg.

[etl@HW04 data]$ hadoop fs -ls /user Found 13 items 
drwx------   - myuser   hdfs            0 2019-07-19 13:53 /user/myuser
...
[etl@HW04 data]$
[etl@HW04 data]$ 
[etl@HW04 data]$ id uid=1026(myuser) gid=1026(myuser) groups=1026(myuser) context=system_u:system_r:unconfined_service_t:s0 
[etl@HW04 data]$
[etl@HW04 data]$
[etl@HW04 data]$ pwd 
/nfs_mount_path/etl/dept/data
[etl@HW04 data]$
[etl@HW04 data]$ 
[etl@HW04 data]$ ls -lh 
total 3.0K 
drwxr-xr-x.  2 882121447 882121447  64 Aug  2 15:34 error 
drwxr-xr-x.  2 882121447 882121447  64 Aug  5 09:49 processed 
drwxr-xr-x.  2 882121447 882121447  64 Aug  2 15:34 processing 
drwxrwxr-x. 31 882121447 882121447 992 Aug  5 09:56 raw 
drwxr-xr-x.  2 882121447 882121447  64 Aug  2 15:47 storage 
[etl@HW04 data]$
[etl@HW04 data]$
[etl@HW04 data]$ hadoop fs -ls /etl/dept/data 
Found 5 items 
drwxr-xr-x   - myuser myuser          0 2019-08-02 15:34 /etl/dept/data/error 
drwxr-xr-x   - myuser myuser          0 2019-08-05 09:49 /etl/dept/data/processed 
drwxr-xr-x   - myuser myuser          0 2019-08-02 15:34 /etl/dept/data/processing 
drwxrwxr-x   - myuser myuser          0 2019-08-05 09:56 /etl/dept/data/raw 
drwxr-xr-x   - myuser myuser          0 2019-08-02 15:47 /etl/dept/data/storage

So notice that the user exists on the client node used (as well as on the namenode), has a home directory in HDFS (and I assume this is all that is required on an unsecured cluster based on this other Hortonworks post: http://community.hortonworks.com/answers/15434/view.html with the addition of having the user exist on the cluster namenode), yet when accessing dirs that myuser owns via NFS gateway, the resulting permissions do not match those shown when accessing via "hadoop fs ..." commands and access is denied.

One example of how this affects operations is that I cannot use the normal FS commands from terminal to delete folders created by myuser. Seeing even further problems in that not even root on the local host is able to delete these NFS dirs or files.

Can anyone explain what this is? Any debugging suggestions or fixes?


This electronic message is intended only for the named
recipient, and may contain information that is confidential or
privileged. If you are not the intended recipient, you are
hereby notified that any disclosure, copying, distribution or
use of the contents of this message is strictly prohibited. If
you have received this message in error or are not the named
recipient, please notify us immediately by contacting the
sender at the electronic mail address noted above, and delete
and destroy all copies of this message. Thank you.