Wrong Root Directory in HDFS API

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Wrong Root Directory in HDFS API

Soheil Pourbafrani
Hi, 

I use HDFS java API to do some process. I pass HDFS configuration file (core-site.xml and hdfs-site.xml) as Configuration. My Hadoop property 
fs.defaultFS
has the value 
hdfs://ha-cluster
but when I get HDFS home directory like the following:
FileSystem.get(conf).getHomeDirectory();
It has the value 
hdfs://ha-cluster/user/myUserName
!!!

What is the reason and how can I set correct home directory?


Reply | Threaded
Open this post in threaded view
|

RE: Wrong Root Directory in HDFS API

Takanobu Asanuma

Hi Soheil,

 

That is a correct behavior. The full qualified path of you home directory in HDFS is "{fs.defaultFS} + /user/myUserName" and getHomeDirectory complements it.

 

You can access paths by a few ways.

======

Configuration conf = new Configuration();

conf.set("fs.defaultFS","hdfs://nn-hostname");

FileSystem fs = FileSystem.get(conf);

 

// your home directory

fs.getFileStatus(new Path("."))  // relative path

fs.getFileStatus(new Path("/user/myUserName"))  // absolute path

fs.getFileStatus(new Path("hdfs://nn-hostname/user/myUserName"))  // full path

 

// root directory

fs.getFileStatus(new Path("/"))  // absolute path

fs.getFileStatus(new Path("hdfs://nn-hostname/"))  // full path

======

 

Thanks,

- Takanobu

 

From: Soheil Pourbafrani [mailto:[hidden email]]
Sent: Monday, May 28, 2018 7:55 PM
To: [hidden email]
Subject: Wrong Root Directory in HDFS API

 

Hi, 

 

I use HDFS java API to do some process. I pass HDFS configuration file (core-site.xml and hdfs-site.xml) as Configuration. My Hadoop property 

fs.defaultFS

has the value 

hdfs://ha-cluster

but when I get HDFS home directory like the following:

FileSystem.get(conf).getHomeDirectory();

It has the value 

hdfs://ha-cluster/user/myUserName

!!!

 

What is the reason and how can I set correct home directory?