[HDFS Pagination]Does HDFS Java api Support Pagination?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[HDFS Pagination]Does HDFS Java api Support Pagination?

wuchang
HDFS-13759

I could use FileSystem

RemoteIterator<FileStatus> listed = fs.listStatusIterator(new Path("hdfs://warehousestore/user/chang.wu/flat_level_1"));

like this to get files asynchronously.

But in fact what I want is a pagination support, where I could pass two parameters, the 

offset

and

limit

, like MySQL does to get part of files under some directory;.

I know I could just implement the pagination by wrapping the listStatusIterator, but I think it is inefficient by wrapping this iterator.

I wonder why cannot HDFS java Api support that?


Chang Wu

Thanks